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ABSTRACT 

Increasingly over the past decade there have appeared 
in the press bar graphs comparing the performance of U.S. students 
With their counterparts xn other countries. Usually these graphs have 
been accompanied by dire warnings and pointing fingers. The maxn 
source of the data on international comparisons has been the 
International Association for the Evaluation of Education (lEA)* The 
articles that make up this document focus primarily cn the studies 
conducted by the lEA. Seth Spaulding sets the stage by looking at the 
history and policy implications of comparative studies in education, 
including those of lEA. Ian Westbury shows how complex the 
interpretation of comparative achievement scores can be. Inge/ 
Marklund from Sweden and Zoltan Bathory from Hungary show how ^wo 
systems of education have used the study results to foster curricular 
changes in the schools of their respective countries. Both show how 
the public and planners car use comparative data in a thoughtful 
fashion. Edward Kifer, Lorin Anderson and Neville Postlethwaite^ and 
Alan Purves examine the various studies to see what broad 
implications can be drawn concerning issues related to curriculum and 
instruction. Herbert Walberg examines the relationship between 
student performance measures and other indicators of a nation's 
well--being in the world. Chester Finn concludes the volume with a 
look at compar^.tive perfcrmance from a policymaker's point of view. 
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Foreword 




he timeline.ss of this btx>k is evidenced by the euiuinuiiig and 



growing interest of educators in international education. 



Whether that interest Ls nxxivau^ by a desire to bener the com- 
mon gcxxl ctf all petite, to improve the aimpetiriveness of our ceo 
numic system, to ensure the survival of our tfcmtKratic government, or 
t«her reasons, it is to bring meaning lo international education and 
glol^ connectedness- 

ASCD S long-range plan carries a mandate to devek>p international 
education a^ivities. A task force erf ASCD members is giving direaion to 
those aaiviiies, and in daily practice we are striving to improve stu- 
dents' understanding of the world in which they live To mode! the 
utgency of the need for international education, the ASCD Executive 
Council has met in countries outside the United Slates, and tlie Associa- 
tion has increased its offerings of professional de\^elopmeni aciiviiies in 
various countries. Each year, ASCD welcomes mentons and new interna- 
tional affiliates. 

Thus, publications such as this complement the person to person 
interaction of ASCD members around the worl i with others interested 
in internaticmal education. This b(K)k moves us toward the goal of inter- 
nationalization of ASCDs progr;ims, services, and governance. To ifie 
extent that it enhances the meiniing of international cduauion, it con- 
tributes to the wonhiest goal: the betterment of mankind. 



Patricia Ctnmm 
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Introduction 



Increasingly over the past decade t^K-re fiave appt^aretJ in tlx: press 
bar gr^i^s comparing tiK* ix?rforniaiKV United States stuiients 
with ih'^ir rounterpans in inl^r counlriei There have been oiY^r 
chaits showing declines or improv^mcniLs by students in txmiparison 
with their elders. Usually these graphs have been accompanied by dire 
warnings and }x>iniing fingers. TIk^ targets have variously bcn^n sdicxjls, 
teachers, television, drugs, race, and parents. As far as the international 
comparisons are concerned, the main source erf the data has been one 
organization, The lntematk>nal AsscKiation for tfK* Evaluation of EdiK^- 
tional Achievement (lEA), of wliich I am tfK- tijrrent Chairman. A yeai 
ago I was asked by ASCD to prepare a volume that would help elucidate 
tiK? various subtle issues behind such a glaring display as a bar grapli 
that shows the students erf a countr>^ like the United Siatc*s performing 
below most dT the otlier industrialized countries. Such is il^ purpose of 
the present volumt% which fcxnises primarily on the lEA studies, T^ hich 
are the olde*;t and most comprehensive of the a)mparative studies, and 
which have been imirated b>' priv^e organizations like the Dallas nevis- 
papers in the early 1980s and more recent^ by tlie National Assessment 
of Educational l^ogress. These studies have been complemenied by 
more derailed studies such as those of Harold Stevensc^n and ihe various 
United States-Japan studies. The lEA studies ser\^% hcnvwer, as a g(xxl 
focal point for discussion. 

In the late 195(hv, lEA sraned as an oigimizarion of researchers from 
around the worid who found that they were concerned with a number 
of issues that could not be studied well within the confines of one 
schoc^l s>'stem. Tlw reason for this is that most systems are more or less 
uniform with respcxt to such matters as class size, age of si Ik)o1 starting, 
length irfthe sclxxrf yean Ciimprehensive secondar\ schtK>ling, and the 
like. If one wanted to stud>' thtfse effects, one nec\led to design costly 
and poliiiailly risk>' experiments. There was, liowever, 'natural varia 
tion" if one ^re to go beyond the Ixirders of a single ge<.)graphic unit. 
The idea of comparative studies of achievement and iLs anteix^krnts and 
consequents was thus born. 
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One of iIk firsi prtA^lcnis ihc group faced was tliai i>f ca-aiing com 
paraMc nKasures, tests tfwi could be used ^ross languages and cul- 
tures. The initial exfx?riments shciwed that this task was feasible, sc> a 
ftiH-scale study of mathematics was launched in the early 1960s and was 
folk)wt?d by tin? "six subject sur\ey" (residing, Uterature, scieiKe, civic 
etlucatkni, and English and French as foreign languages) in the 197Qs. 
During the coune ot the past dec^% lEA has amduded a second s^udy 
in niaihematkrs and science, a study of written a>m|H>sition, of c la*^- 
nx)m environments, and Is currently a>nduaing studies of preprimary 
education and computers in education. It is launching a study of adding 
Iiterac>; the first in a series of projected cyclic studies of learning in tlie 
basic school subjecxs, and is contemplating a stud>' erf scxial values and 
moral reasoning. 

In general, lEAls iricthcxlolc>g)' is one of survey research, with an 
emphasis on careful test construction combincxi with sets of question- 
naires for stutfcnts, teachers, and schcx)! iK^ads, as wx^ll as "naiionaF* cur 
ricnilum questionnaires and supplementary^ histories and inter\iews 
Over the course of its histon; lEA has use-J various approaches tc^ the 
analysis of data and lias been among tlie pioneers of various scnts of 
causal rmxleling and analysis. 

In some systetns of education, IRA has come to be seen much as 
people in tlx* Vniwti States see tlK* "wall chan ' pitting stale against slate 
on the basis of :>AT scores. Uke SAX lEA is a household m)rd, bc«h 
feared and respetied and even cherished, depending upon the educa- 
tional stripe of tlie viewer It lias bc*en seen as creaimg an educational 
Olympics. At the same time, those involved in tlK* woilt of lEA have 
always been char\^ of the simple comparisons; they have downplaycxi 
the Simple comparisons t>f mean tc*st scores for a number of reasims 
which aa* the sutistance of this volume. In summary^ scIkkiI systems are 
often struaurally different and any comparison must be seen first in 
light of such structural differences as percentage of the age cohort in 
school, amount of prior schiX)lif)g, tracking and streaming practices; 
second, in t<:rms of the subtle curricular differeiKes that lEA has labeled 
"opfX)rtunity to Ic^inV*; and third, in terms of a*al cultural differences in 
conceptions of the given subjeti matter and irf schix>!ing in general 
Each of these issues is addressed in this volume 

Seih Spaulding sets the stage by kx^ing at the history and jx)lic7 
imp!icatk)as of comparative studies in educatk)n, including those of 
lEA. He elucidates ihc various theoretical and practical issues in com- 



parativc studit-s and ir;Kc-s t^K- st>ns <>f intt-rprciive f rohk-ms rlwt IKA 
and tHher comparative siiKiics have latcd in conipai Mig ihc iruumpara- 
Me. 

Tbrning to sptxific isMit-s ^it what lo make of a a)mp;iris<)n, Ian 
Westbury shows how complex nvdy Ik- the iiiierpreiation trf comparative 
achicvenK-nt scores. Using the recent niaiheniatics data as an example, 
he shews that tlx- comparisons rai.se a number of alternatives in curricu- 
lum planning and sc1kx>1 orRani/Jtitm. Any simple answvr ma>' be mis- 
leiKling. 

I>?spite thiK caveat, the lEA studies have spurred educational 
reforms: Inger Marklund from Sweden and Zolt;m Bathorv from Uun- 
Rar>^ sIkw Ikw two systems t)f education have used iIk- results to ft)sier 
curricular changes in the sclnxils of tlwir resfmiivc countries, lloih 
slKW h^nv tlie public and planners can use comparative ilata in a 
thoughtful fashion. Their description of how tlK'ir systems used ilie data 
are potential guides lo curriculum planners at the district and state level 
in the United States. 

Edward Kifer, k)rin Anderstm and Neville l\)stleihwaite. and 1 
examine the various studies to see what broad implications might be 
drawn concerning Issuer relatt\l to curriculum and insiruaion Kifer 
pays particular -^Ttention to issues relateti to the soning of students as 
well as to {«hcr issues of schix)! organization such xs retentiviry selei- 
tion, and the differentiation t)f the curriculum. TlK-se all have their 
effea on opponunli>' to learn and thus on achievement. Anderson and 
festlethwaite lotik at the lEA results in terms of cliaracieristics of teach- 
ers and teacher training and at ckssr<x)m arrangement and teaching 
praaices, They draw some general conclusions about the curriculum 
that affea the students and their teachers. In tracing the panicular impli- 
cations for a single area of the curriculum, language arts in the mother 
tongue, 1 examine the results of the reading, literature, and written 
composition studies to see what implications might be drawn f<ir 
reform. The studies m reading and literature had an effeii similar to 
that of the mathematics and science studies; they sent researchers and 
teacher trainers to a panicular anmtry, New Zealami. to sc>ek to deter 
mine why the students there appeared to be far better readers than stu 
dents in other a^untries. That experience has led to profound shifts in 
the language aas curriculum, panicularly at the elementary level, 

Herben Walberg follows with a provocative examination of the 
relationship between student jwrformance measures and other mdiv-a- 
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tors of a nations \wH being in tin.' world His chapter sets ti>nh ihc idea 
that educational performance can be a harbinger irf et\inc.«ic \^'ell be- 
ing and prixluciivity in an a^' of Information and techm>Iog\'. Il is this 
ptwerful link that Itas helped spur the sons of stud*'^ that lEA repre- 
sents. Although the other chapters suggest that gross indicators mask 
many imjx>nant nuances, Walber?; reminds us thai gross indicators 
uscxl and will ct>ntinue to Ixr used by policymakers around a world that 
is increasingly inierconntxicd and ik>pendent ujxmi a highly educated 
workforce. 

Chester Finn cxincludes the volume by kHiking at ctimparativc per- 
formance from a policymaker's point of view. As his chapter indicates, 
during his k-rm in the Unittxl States l)epanmcni of Educatit)n, Finn 
pushed for ihc development vS clear and understamlablc txlut-ational 
indicators that could be ased much like tvonomic indicators to show 
the changes and a*lative position of txlucational systems Such indica- 
tors represent a part of the |->t)iicni;iker's mx\l to rej^ort clearly and sim 
ply to the public. He places international studies in tlx' context of 
national and state-by state comparisons Tliis paper makes a plea for 
continued comparative indicator studies, particularly among the indus 
trializctl natitins Such studies, including the lEA studied, are an imjitir- 
tant aid to the etlucational dtvision maker, whether profes-sional pi>lic> • 
maker or informed ciii/en 

AlAN C PtIKVFS 



Comparing Educational 
Phenomena: Promises, 
Prospects, and Problems 

SETHSPA11.DING 



We all k-arn frDni ajmparist)nN. and ifn- history irf tomjianns 
and btjrrijwins ideas in ttlucation. auording u» liiMoriaii/' 
CDmparativv cxhK~«(jr Bill Brickman { 19KH. in an aniiic 
publishetl just before his recent death), can »e iractxl at least \a tin* 
founh CL*ntur>' B.c:, w!H»n, i.i QnifHwdia, Xenoptjon analy/iAj what he 
described as IVrsian education, thereby ena!i!ing tin* (Jrevk reatler to 
compare it with Spanan schixilin^ " (p, 3) Indeed, tor over iw) millen 
nia since then, until relatively nnxlern times, comparative studies in 
etlucatitm were es-sentially reports of scholars. >{overnment ofticials. 
and otlK-rs wlu) described tlie txlucatitinal institutions, systems, and 
pri)grams they .saw when traveling. 

With the ft>rmalization of comparative education after \X/brld War 11. 
scholars began to use more strumired ajiproaclK-s l>orrt)wi.xl from vari 
ous MKial sc ience disciplines, conceptual fra>.ieworks and methiKls that 
provitk.*d quantifiable elements to compare The questions were, esscn 
lially. how to cumpare educatii)nal systems and pheni)mena so that we 
know we are kx)king at Mimeihing truly < omparable antl. at the same 
lime, disctiver reasonable explanations about why things hapix-n as 
tlvfy do To some scholars, such a)mparisons were primarily ()f intelkx • 
tual interest, part of tlx' drive to kiK)w mt)re atx)ui other culiua-s and 
pe^jples. Others, panicularlv in iIk' I'nited Kingdom and KurojK-. 
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SeU) y^patiUiin^ is I'nj/i'ssor nj lutetymUofkil uuU Ik^t t ki^ymeni liduat 
turn, t>ef}artnwnt cjf MimmLstn4tiiv mid SdH>vl<f/iciuca 
Hon, Unmnsit}' tj^ l^tLsfnif}>h 



A. 



llKiught |x)lk7niakiT.s could learn from ilw cx{>criencc u{ oilier eoun 

t:oniparisons in ctiucauon, whaioer the nn^ivc, lake pknv in a 
wmplex eiivirunmem. c;hiidren in differcni eoumries attend mIkxjIs in 
sutetanttaMy diftcrcm Mxieiies and educational systems. TIk'v aitetnl for 
van ing peniKis i)f tinK% both during t!ie scIuk)! >var and during tlK*ir 
cumulative scIkxiI career Some systems are oriented toward central- 
izetl dcvision m;iking and national ex;imjnations to select those to con- 
linuc* Ix^yond compulsory Ic\ds. C)ilK-rs are highly decern rali/ed with 
little sundaTxli/x\i achievement testing as a l>asis for pronKXing stu- 
dents. Si>me countries retain nnisi of their tlnldren in scfKxi! tlirough 
the stvondar>' level; many still do not provide a full primary sc1kx>1 edu 
cation for ever> vhiUI. 

At tiK* same lime, tlie eduational problems and issues between 
and among nations are irften very siniilan essentialh' revolving around 
Ikw to provide education that is tleni^Kratic. i^fTicient. and rek vani, h 
seems ajiproj^riate. therefore, that try to learn from onv another 

Sharing Descriptive Information Among Countries 

The l^niitxl Nations Educatii)nal Scientific Organization (rNKSCO), 
established in 1946, early recognized tlx: need for sliaring of informa- 
tion about educational systems imiong countries Hcmever, statistics col 
letied from member states did not to report comparable phenomena. 
Age groups in one countr>' were slightly different at varying levels or 
grades than in other countrit^s St)me countries re<}uirai more years ai 
various educatitmal levels than others Some rc|>oried exjX'nditures in 
categories not uscxi by others, and, of course, curriculum emphases var 
ied from country to ci>untry 

Over the years. TNKSCX) develojxxl, in coo}HTation with etluca 
tional authorities in memIxT states, the International Stan^brd c:i;issifi 
cation of Education (ISt^ED) that provides agreed upon siandirds and 
definitions so that we mnv liave a better chanc c of untlerstanduig what 
statistical data fn>m various countries mean. ISCH) is dc*signed for 



^Scx* En\'in Epsleln, "CurrentN U-h ami Kighl Ulei»U)g> in Comparative 
EducatiiHii ' in Netv Afjpnxnines m ComfHirtiinv Fsiixiitkm, itlittxl by PG Alilwk 
and G. Kelly (Chicago: t ^nivcrsity of Chicagi) Pre a, 19S(>>. for a fascinating dis- 
cassion of the varit>us idLt)it:>gical aixi mellKxlok^ical currents m the fieUi 
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assembling »jata on current ediicatuinal phemimena such as enrollment, 
teaching staff, and finances, as well as u»r statistics d[ the "sttKk" of edu- 
cated people f*tained, for example, by a national ceasus. It classifies 
"(xnmeSyprogmmmes, and fields of educati<>n according to tlieir educa- 
tional content. The educational content of each couree, programme, and 
field is designated according to its tetirf calcgor}> and its sutyea tnaiter."^ 
h\jrtljer refinements make it possible to intelligently compare statistics 
on education from various countries, as long as they use ISCED. Fonu- 
nately, most do, or at least report data st) that they can be regit 'jped fol- 
lowing ISCED standards, and DNESCO's statistical yearbook is an invalu- 
aWe source of information. TTiese statistics are often used in conneaion 
with other audies, such as the ratw)f-reiurn, cosi-baiefii, and man- 
power studies often done by ea)rK5mists, and tlie «)gnitive achievement 
stuilies done by groups .such as the International Educ-ational Achieve- 
ment Association (lEA). 

The availability of better statistics, t>f cour.se, has helped to describe 
wliat is happening in various countries in terms of enrollment at vari- 
ous educational levels, broad curriculum trends, dn)p-iiut and wastage 
proHems, p»jucational finance trends, and st> fc)rth. but these data have 
only illuminated tte problems without much information about solving 
them UNESCO, accordingly, encourages sharing qualitative informa- 
tion through documentation center work, both a' UNESCO headquar- 
ters in Paris and at its International Bureau of Education in Geneva and 
through a variety trf conferences and studies. Ptrh^s most notable is 
the International Conference on Education, held every two years in 
Geneva, wl:-wh attracts ministerial dele^tiom from over 1(X) countries. 
Before the conference, each delegation responds to a questitinnaire 
abtjui its theme, and the results are summarized in conference docu- 
ments. In addition, most countries submit a report that is microfilmed 
and made available to researchers and c\iucators. Unfortunately, the 
Keagan administration withdrew from UNESCO in 1984 Since then, the 
U.S. has .jot panicipated in nxwt UNESCO educational efforts 



UNESCO, iTUemational Standani OassificaHon (^Education, abridged 
edition (ED/BrE//CONFliYrED.3*VRef8) (ftirls, July 1985), 1 and 21 See aim 
"Educational Statistics: National and Intematj<.)nal Sources and Services," in 
Editcational DKXumentation and /nformait • (Bulletin of the International 
Bureau of Educatkm, No. 202, 1st Quaner 1977), which dt-scrlbes in detail the 
UNESCO statistics collection system. 
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The LVilqiie Role (rfthe lEA 

UNESCO IS the major intergovernmental organization resrK>nsible 
for sharing educational information v^'orldwide; numerous t«her gov- 
errjnental organi2atioas are involved as well. The World Bank, the 
Organization for Economic Cooperation and Develt^ment (OECDX the 
Or^i- -on trf American States (OAS), and various bilateral organiza- 
tions s, as the U.S. Agency for International Devekpment (AIDX 
Sv«xlish International Development ^ncy (SIDAX Canadian Interna- 
tional Development Agency (CIDA). and the U.K. Overseas Develop 
ment Autlwrity (ODA) continue to suppon various kinds trf^educatii^nal 
aaivities in a number erf countries. None of these agencies, however, has 
attempted to coordinate a study as comptex as tht>se undertaken by lEA. 
Such studies have been passible only l^caase of a uniqMe process df 
collaboration among researchers, private and public iastitutions, and 
governments. 

lEA and its wori< have grown from modest beginnings at a meeting 
of educators, psychologists, and curriculum specialists at the Hamburg 
UNESCO Institute for Education in the late fifties. They feh the need for 
more concrete information about what stuctents around the world were, 
in fact, learning, especially in ^ademic areas. Some felt that such infor 
mation would be useful in identifying the possible effects of differences 
in education?J systems; others simply h^l a scholarly interest in finding 
ways to measure and compare C()gnitive achievement among countries 
d" different cultures and languages. It became, as the eminent American 
comparative educator George Bereday said to then lEA chairman 
Tbreten Iius6n a few years later, an attempt to compare the incompara- 
ble. 

AltlKJugh viewed by some as a kind of cognitive Olympics, the lEA 
national achievement rankings have often been used by critics of educa- 
tional systems and by those who urge educational reform. Clearly, more 
and more countries and educator; in those countries are intrigued by 
the lEAls woric, as the number of participating countries rose in recent 
years from 20 to some 50. Both the Soviet Union and China pt>ssibly will 
participate in tte future. 

But the lEA studies are more than a cognitive Olympics. In a very 
real sense, they represent tlie frontier of comparative research in educa- 
tion. The studies anempt to answer questioas raised by policy i roups 
both nationally and internatfonally about what learning educational sys- 
tems and institutions produce and at what cost. People worldwide are 
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searching for indicators of educational effectiveness and efficiency 

A number of scholars see ihe lEA studies as an imix)rtam cximple- 
ment to oreHX>untry studies such as the U.S. National Assessment trf Edu- 
cational Progress (NAEP).^ As NAEP has moved in 20 years from a con- 
cern for achievement levels to a amcern for variables that may be 
related to achiexx^ment,"* so lEA has moved vi^roasly into the an^ysis 
of these variables ur indicators. Unfortunately, methoctological difter- 
ences exist between the two approaches, Anwng others, NAEP examine 
students on test items seleaed by a panel outside the educational sys- 
tem, while lEA uses curriculum-based tests. 

Indiottors, in educational policy research circles, are essentially 
statistically demonstrated relationships Ix^rween cognitive achievement 
and content, structure, methods, teaclKrr char^eristics, and/or organi 
zational processSes within tl^ school and/or scxietal faaors outside the 
school sys.em. The prc^lem, erf" coun^*, is whether such rolati<inships 
suggest cause and efFc^ and, if they do, whether we ran prescribe 
improvements based on these relationships. 

Methodology Issues' 

lEA so far lias used essentially surveys to gather data, and all the 
problems of such a metiKxiokigy are multq^lied in a multinaiion effort. 
The AsscKiation is largely cfcpendent on iht* good will of participating 
institutions and researchers, and methodological issues ha\^c been 
resolved within each project by discussion and agreement among the 
researchers involved. Some of these issues are truly formidable, csfxfci- 



^ See, for example, Richard M. mAf, TTie NAEP and International a)mpari- 
soxi'ir Pbi Ddm K^pan 69 (April 1988) 5a>-581, 

"^See, for exanple, Daniel H and lauren K. Resnick, Undemanding 
Achievement and Aaing to Prcxim.'e It: Some Recommendaiioas for the NAEP; 
PbiDetiaKofpan 69 (April 1988): 576-579. 

^The fdlowing researchers and educators affiliaied with lEA pmjecis gra 
ciously respomfed to my request for suggestions and awnments on the issues I 
wished to explore. Oniy lack cf space prevents a more cun^kte inclusion of 
their richly detailed responses. They are RIcardo Choners dAzevedo (ft^rtugalX 
RA Garden (New /ealandX Jack Holbroc* (Hong KongX Tbrsten Hus^n (Swe^ 
den), Inger Markltnd (Sweden), ftnom Pongpalbocrf (Thailand), Neville 
Posilelhwaite (GennanyX Alan Purves (USA); Vivten M. T^Lsayon (FliilippinesX 
Aneli VahapassI (Finland), Mkrh^i Wilson (AustrallaX and Richard Wblf (USA> 
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ally when aticmpting to decide what is relevant ft)r testing in each coun- 
try' and at each grade level. 

The lEA participants and informed observers alike engage in lively 
and spirited dialc^ue ab<iut both tlie process and the metlxxiologies 
used. Keeves, for instance, notes that the early ^udlcs U^ked a clear the- 
oretical framework, and at the same time, wondei^ if, in certain kinds of 
projects, tlie dcvebpmeni and use of a coherent tlieory are impossible. 
He also notes the time and resource? constraints for getting reseaat 
panicipants together and for dc^velopitig common promlures and sta- 
tistical techniques. He comments that the working group involved in 
setting the aims and objectives of the Second lEA Mathematics Study 
was dominated by research workers and curriculum specialists from the 
United States, Vho apparently cannot conceive of very different 
^proaches to the teaching of mathematics than are en)plo>^ in their 
country" (Keeves 1980, p 8> He also notes that the earlier studies 
involved a gbbal strategy for examining the data and did not, in gen- 
eral, seek to explore specific relationships and to test specific hypothe- 
ses (19«0, p. 19> 

Marklund (1983). in turn, suggests that curriculum and research 
methods arc constantly changing in many countries and questions tlie 
longevit>^ of reseaah results. He says that when sc*arching for indicators 
which may have some relationship with schcx^l achievement, disaggre- 
gating the data is important so that one c^n see what hapjoens in indhid 
ual schools and within various jx^pulation groups Pustlethwaite, 
involved in lEA from its beginning, is fully aware of the ct^mplexities of 
comparative study He notes that lEA methock:)k>gy has matured and 
mw most projecxs not only colleen achievement sabres, but also con- 
strua and apply anitucte scales and b2N:kground information questiK^n- 
naires. 'The background questions obtain information from students, 
tethers and school principals — indicators which can be used to 
explain diffcornces in achievtriiient outcomes 

Although lEA projeas began as siraightforwaixl comparative 
achievement studies, they soon evolved into effective schools 



Neville Pasilethwaite, "Cross National QMiveigence of CX>nceois and 
Measurement Educaiional /^xhicvement ' (paper ctelivcred at the annual 
meeting of the American Educational Research AssiKiat ji)n. New Orieans, April 
1988). current s^histlcation crfthe lEA studies is illustrated in the recently 
issued preliminary^ lEA report, written by ftistlethwaite, t>n Science Achietv^ 
merit in Set^een Countries (Oxford: IVigamon Press, 19^1 
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research." "Effective scIkx>1s research" usually focuses on what happeas 
in individual schools, and such studies to date comx?nt rated on elemen- 
tary schcx>ls in urban areas, lEA, on the other hand, deals with both ele- 
mentary and secondary levels in both urban and rural j^ntings and has 
rqx>^e^:1 aggregate data by country. However, cunxmt lEA and lEA-relat- 
ed studies are increasingly disaggreg^ing school-level data. 

Heynemiui of the World Bank (an organization that stresses quanti- 
fication of indicators so it can assess tte impaa of its invratment in edu- 
cational envelopment) suggests that the availability erf comjTulers b<^in- 
ning in the late 60s made possible the identification, measuren^t, 
control, and analysis irf "hundreds of influences on school achievement 
simultaneously" ( 1986, p. 3). The Coleman report was an earlier attempt 
to do this on a national scale in tl^ IIS., and its results disturbed educa- 
tors. They felt that the effeas of schooling were minimized by 
Colemans methcxlolc^y Bowles and Gintis point out, for iastance, the 
dangers in construaing a model in which personal influences were 
forced into a regression before other variables and suggest that ux> 
many variables were coJinear and insufficiently aa-ounted for (cited in 
Heyneman 1986, p 4). !leyneman cites several studies which suggest 
the,, in many less-developed a>untries, little relationship exists 
between stxioeconomic status and educational attainment, although 
studies in advanced countries shew that they do make a difference. 
Clearly, nietliodological issues are n(A yet entirely resolved 

Further Comqplexitles of Interpretation 

Effective schix>ls research attempts to establish the functional rela- 
tionships "that exist within and among the four stages of educational 
produaion — inputs, pnxess, outputs and outcomes ' (Windham 1988. 
P H9> Figure M lists some of the taaors in what economists call the 
"education prcxluaion prcxess. ^ In such a mtxlel, eiatwrate attempt-^ are 
made to determine the relationships among the varioas fxtots. In addi- 
tion, the usual assumption is that the various inputs and the ways they 
are put tc^ether (process) are determifumts of the outputs and out- 
comes (the effects) Such mocfels do not deal with qualitative issues 
involving the relevance trf the otaptus (cognitive achievemeiit, skills, 
anitudes, l^haviors) to such goals as empbyment, earnings, and social 
status. Thus students might thieve academically, but be complete mis- 
fits in terms of many societal and/or individual needs, EvSsentially, mod- 
els such as Figure 1 . 1 largely ignore the political, economic, and cultural 



ngure 1.1 

Major Factors in the Education Production Process 



CETERMINANTS 



Inputs 



Process 



Student characteristics 
TeacJ^r characterislics 
Sch<x>l Characteristics 



Forms of instructional organization 

Alternative technok^ies 

Use of teacher and student time 



Instnjctional matorials and equipment 



characteristics 



EFFECTS 



Outputs 



Out<x>mes 



Cognitive achievment 
Improved manual skills 
Attltudinat changes 
Behavioral changes 



Empioyinent 

Earnings 

Static 

Attitudinat changes 
Behavioral changes 



SoufToe: Douglas M Windheun, ^Improving ttie Effjctency Edi^atiaial Systems Indic^cKS oJ 
EdiiC^tonal Etfectiveness and Ef1ic»efx:y" (paper presented IFES Educational Eflicjency Ctear- 
^>gh0iw. Learning Systems (nslilute, Tallahassee, F^tda, January 1988). 9 

systems liiat contain tlK* educational sellings and may provide more 
powerful explanations for what is happening i \ the schools than do ihe 
faaors in the model 

Marlaine E. Lockheed of the World Bank nc^es that such a model 
**must able to separate the influences of the different inputs to the 
educational process in order to judge their effeaiveness " the ccs^ts of 
each must be determined, and all of this must be interpreted in terms erf 
the way an eduational system uses inputs in vartous settings (Lockheed 
1988, pp. 12-14). Lockheec a>gnizes the difficulties in undertaking 
such polic)' anal>'ses \ itnes, for instance, that educated parents in 
ctevetoping countries ^ V to provide lining in ilw home and 
other rescHircts and mhO to ; ^d their children to schools with better 
r«ources and better trained te^^d^^rs. In a pereona! letter (May 22, 
1988), Jack Holbroc^ points out that sock>economk: level can affea 
achievement in Hong Kong because students from wealthier families 
have accesr to tutoring. Illiterate parents, in turn, are likely to send tteir 
children to schools with lesser resources. How can tte influence of spe 
cific school inputs be separated fmm each other or from that of parents 
(Lockheed p. 12)? Funhennore, the use of inputs is difficult to 
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determine. AbJy used textbotiks, for instance, may shtnv a sizable effea 
on achievement, while poorly ased (or badly consiruaed) texts may 
sIkw no effect 

Several critics erf the lEA methodologies take a similar position: that 
the lEA studies are significant but that the explanations of variance, 
using the limit»J information collected, arc questionable. Fut'ire 
dforts must be ttevc^d to contextualizing the dao we are generating in 
a meaningful way," suggest Theisen and his coauthors (3986X who list 
various "educational produakMi funaton" studio in poor countries in 
wiiich fwrtors related to adWevement conflkt. A critique of Hieisen's 
study, lM3wever, suggests ih^ the lEA studie;, plus the broad range of 
^In-cff natkmal and internaitonal studies, are in (sKrt increasingly exam- 
ining local j^nomena,"^ though even these authors sug^ that "ttere 
Is no aatiaical ^roach that is even minimally acceptable in ruling out 
the many alternative explanatkjns for the difference that are observed" 
(Schwille and Burstein 1987. pi 602)i ^rld Bank specialist Steve 
Heyneman (1986. p. 1), on the other hand, sugg^ ihai many educa- 
tional i^enomena are similar from coun' y to country and culture to 
cuhurc, and that concerns for differences sliould not obscure the simi- 
larities. 



Vdlkry Uses and Abuses of Gonqsoratlve Studies 

The comparative study of educational phenomena, it apf«an», is 
now an item on the agenda of chiefs tjf state. "When Prraident Reagan 
and Prime Minister Nakasone met in 1983, one result of their discus- 
sions was an ^reanent that a cooperative undertaking by Japan and the 
United States to study educai on in each other's country would be 
worthwhile" (Leestma et al. l^ST, p. vii> Each country formed a task 
force to study the education sy«em erf the Mher, and the US. repon (a 
kind trf case study) on Japanese education was published in 1987. lEA 
figure showing that Japanese achievement scores in mathematics are 
higher than in the US. undoubtedly motivated, in part, presickntial 
interest in the studio 



'' Recent aamples include A Origen SocioecononUca do Mum e o Sucesso 
Bcc4ar (U*on, Portugal: Ministerio da EdiKacK), February 1987) andDetermi- 
norm l^ffective Scfwofe Megioml Syntheses (Bargkok, Thailand Ofrice of the 
National Education Commlsston, May 1^7> 
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In an epilogue to the report on Japant^* education, tlie then Secre- 
tary of Education WiHiam J. Bennett lists what he corusiders to be impli- 
cations for American education. He chides American educators who, he 
feels, *'have tended to shun the 'lesstms' crfjapanese education" (Lceitma 
et al 1987, p. 69). He lists twelve 'principles'* that he gleaned from the 
jitudy and u^ich, he feels, agree with U.S. research findings: the impor- 
tance o( parental engagement, clarity erf purpose, motivation, ex^-*"' ^ 
tions and standards, comprehensiveness and quality, charaaer an 3 v^l 
ues, wellorckred school environments sufficient time on task, 
sufficient resources, good teachers' salaries, and youngsters who take 
responsihility for their educational achievement. He takes his usual 
swipe at Colleges vS Education: "It may be noted that, in most cases, the 
Japanese do enter tlie te^hing prcrfesston via colleges of etlucation, 
nor is it necessary to do so in order to Ixr knowledgeaMe about ones 
field and competent tr^ transmit ones ktxwJ^gc to young people" (in 
Leestma et al. 1987, p 71 ). 

Some erf Bennens generalizations are somewhat disturbing and 
illastrate the question^le interpretation irf comparative data. For 
instance, his implied noticMi that mxxld well eliminate prcrfessional 
education courses when preparing teachers is a clear misreading trf the 
data in the repi>rt. Although most J^anese teachers di> rKH attend a>l' 
leges of education, per .se, they are required to have substantial prepara- 
tion in professional eduction subjects to otHain a teadiing cenificate. 
These subjetls are offered by colleges and universities that prepare 
yioung people for teaching certificates even though there may not be a 
formal College of education" in each institution. The minimum 
requirements for a preschool or elemenmry teaching certificate include 
about twice as many credits in professional education subjects as in 
teaching subjects, and minimum requirements for a secondary teaching 
certificate include from about onequaner to onc^half prcrfessional edu- 
cation subjects (cfcpending on the level and class crfc^rtiticate). This is 
haixUy ammunition for Bennett's crft-repeated criticisms erf professional 
education courses (Lresima et al. 1987, p. 16), Furthermore, a complex 
system of inservice teacher education centers continually ujxlate icxh- 
ing staff in both content and meihcxi (Leestma et al, 1987, pp. 17-18> 

The rosy picture paintc*d by Secretary Bennen erf "lessons ' to be 
krarned from Jaf^anese education is furtlier eroded by a mimeographed 
document originally prepared by the Center for Education Statistics of 
the Depanment of Education to accompany the report on Japanese edu- 
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cation, but as of mid- 1988. issued only in draft fonii (Suter n.d.> Much of 
this rep'jn builds on the lEA math srudy and related «udies in Japan and 
the U.S. It points out, for iastance, that altlxiugh math achievement 
scores in J^an seem higher than those in the II.S , Japanese students 
"are nt« necessarily likely to express a high interest in the sul^ect. Japa- 
nese students are less likely than U.S students to say that mathematics is 
important or fiin , . for) ... that they j^rceive the utility of learning 
mathematics for the future . . ." (Suter J 988, p. 11). 

In atWition, Bennett's lessoas from the Japanese study suggest that 
we should learn from their i^pro^h to parental engagement and the 
way this contributes tcj motivation. The statistical analysis of J4)anese 
and U.S. survey data, hovvever, suggests that "Japanese students perceive 
less parental encouragement than do U.S. eightli-grade students" (Suter 
1988, p ll) 

Bennett also laurJs the Japanese s)^em for providing more time in 
school and more homeworit, on the whole, than we do The results ctf 
the Department's statistical study, however, suggest "that the dififerences 
in use of class lime and amount of hours of homework can « explain 
the differences in students' achievement between the two ^-ountries" 
(Suter 1988, p. 291 

ft;riiaps more significantly, some features of the Japanese education 
system from which the Secretary gleans his "lessoas" are those which 
the Japanese tltemselves question. The same repon, just before the Sec- 
retary's epik)gue, notes that an outside review by the 24-nation OECD 
some yeam ago lauded J^^an's educational accomplishments, but criti- 
cized the extent of "ceniraliEed amtrol, standardization, conformity, 
institutional hierarchy, and the emphasis on university entrance exami- 
nations" (l4:estma ei al. 1987, p. 63). 

The repon alsMj noted that the achievcn^nt ctf Japanese 13-year-old 
f in the second (1981-2) lEA mathematics study declined since the first 
study in 1964, and that there is an "increasing number of low achievers 
in primary and secondary sctools; increased school violence, especially 
at junior high schools; and tlie emerging phenomenon of voluntary 
dropouts from senior high schools" (Leestma et al, 1987, p. 63). A 
reponed decrease in public txinfidence in educaiton led reformers to 
concentrate on enforced uniformity "that is believed to stifle individual- 
ity, create frustratioas and contribute to disorder in schools, and on the 
.heavy emjAasis on uni\^rsiiy examinations which is believed to hinder 
personal and intellectual development" (Leestma et al. 1987, p. 63)l 
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Japan apptnmed, in a National Council on FAiucational 

Reform that issued two critical reports sugRCisting that the rigidity of the 
system created prt^blems such as 'buUying, schot)! violence, juwnile 
delinquency, and \he refusal to go to schix)!" and that the rigid, uni- 
form school programs, excessive aintrols on studwits, and other faaors 
prevent sound charaaer formatkin, increase pressures on childa*n, and 
create frustration." FUnhermore, the Council "fears that an excessive 
emj^iasis on memorization has produced many conformist children 
who are unable to think creatively" (U-esima ei al. 1987, p i,*)). The 
report notes that the current reform movement in ppan is moving in 
the opposite direattw from that in the United j:iates. Educ-ation reform- 
ers in J^?an are seeking some dcanitralization trf control, greater diver- 
sifkation of iasiituiioas, less uniformity and standardization of curricu- 
lum, more flexibility in teaching, and more individualization in 
instruaion (Leestma et al., p. 67). 

The sharply chiseled "principles" that Bennett derives from the 
Department of Educaiit)ns study of Japanese education leave something 
to be cfesired. Indeed, Japanese students in tlx.* aggregate .seem it> do 
somewhat bener in math than U.S. students, but at what price?** 

The difficulties in applying "priiKipkrs" from other countries are 
identified by KA. Garden, Director of Research and Statistics, I>epan 
ment of Education in New Zealand, and lEA General Assembly repre- 
sentative for his country, who, in a personal letter, suggests that 'what 
^jpears to be a growth industry for U.S. educational researchers, ctim- 
paring the USA and Japan is likely to be fruittess |in terms of suggesting 
policy reforms). Fetors which influence opportunity and mixivation to 
learn are tcx) different in thc.se widely differing cultures." 

Ideology, Interpretation, and BorroTving 

Former Secretar>' Bennett's interpretation erf the J^anese c^- 
stud>' is a clear dcmoastration of ideok^ically .'elective interpretation of 
data. But seleaive interest in education and research acrording to parti- 
san or sectarian views is not unusual. Under the Johnson admini.stration, 



*• Harold Noah tkxruments other .such mlsases of data In liis emincnily anid- 
aHe artJcJe, The Vses and Abuses ot Qmparative Educatitm." Compamtiiv 
Edtaatkm Retiew 28 (November 1984): 3S0-362. 



for instance, racial iiiequities wt-re trf prime impt>nance aiHi renected in 
the research and mlm federal prt)grams tlien enc<iuraged by the 
administration. Under t\w Reagan administration, other educational 
conceras liave taken precedence (SchwiUe and Bursiein 1987, p. 605). 
In an excellent paper on the complexities of using lEA data in policy- 
making, long-time lEA chairman Torsien Husdn (1987) iK«es thai ihtwe 
in England and in Germany both for and against greater ct}mpa'hen. 
siveness in education ased the same Swedish studies to suppon their 
positit)as. 

Comparative educators, t>f course, point out that the initial choice 
of a research paradigm (the currently fiashionabkf word for strategy or 
mtHiel) is heavily influenced by ideology, Many researchers in tximpara- 
live education adopt such concejwual frameworks as dependenc)' the- 
ory, which assumes cvnterperiplwry relationships vwrldwide, and, 
more generally, conflia ilicory, which suggests that tinkering with the 
educatwn system will do little gtxxl in achieving economic and stKial 
equity and justice unless and until there is a revolutionary change in 
fKrwer relationships within s(xritties.^ 

Amove (1982) suggests that bt)rrowing txlucational approaches 
around the world simply reinforces various kinds of economic and cul- 
tural dependencies. He argues that a world system of education exLsts 
within which nations both at the center and {>n the periphery' are domi- 
nated by a few of those at the center Indeed, it is difficult to identify edu- 
cationaJ struaures or innovatioas that have been bt>rrt)wed by rich 
countries from ptwr wunirics, but it is easy to catalogue the features of 
educational sv.stems of rich countries that have been iastalled (.some 
might .say often inappnjpriately) in ptx)r countries. 

In faa, successful innovatioas we have boramt-d from abroad 
often have little to dj with solid research evideiKC- Many are simply log- 

approaches to .solving technical and pn>fessional proble.ns. <ome 
have proven useful to the borrowing toujitrv, >omt: have not. One posi- 
tive example is the Open linivcrsit)- in the llnitc\i Kingdom, whose 



For an infomiativc inenicw of trends in ainiparaiivc .studies in educa- 
tion, sec Philip I Mihtch and Gail KelJy in New Appniacbei to am^tmtife 
Education (C:hlcago University of Chicago Press, 1986), Tor.sicn Husen, 
"Research Paradigms in Education," IntercJx4n^ 19 (Sprii^ 1%8): 2-33; and 
Carkjs E. Olivcra. "Qimparaiive Education: Toward a Basic Theim " Prvmia 18 
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appnmh to distance higlier t>ducaik>n tus btt-n replicated and adopted 
througlviut the world. Another is the Teachers' CX^nier cxintqit pio^ 
ncered also in the United Kingdom. C">tlier examples include literacy 
methodologies pioneered in oilK*r muntries and used successfully in 
di^mestic programs. 

When T\'e aimpare and, as a result, btwrow, many quesiioas sur- 
face. Frequently borrowings are huscd on intuition or the word of 
"authi>rities." however defi x-d. The "new math" of the early '60s is an 
example. It was deveksptd by mathematics scholars cDnvincxxl that 
advanced mathematical tlieory and thinking should be tauglu early 
(before computational skills) to facilitate the Irarning of higher math in 
later years. With link! research evidence, "new math" was accepted (with 
minor mcxlifications) by a^untries thnjughout the world, including 
many ptxjr counirk-s, starting with the S4>-calkxl 'Entebbe' math, 
launched at a meeting in Entebbe, Uganda, in 1963. Now, years after the 
lis. has largely abandoned the "new math, ' many other nations, espet i- 
ally in the developing world, are still struggling with ii. Many teachers 
and siuifcnis dti mrt fully understand it. and it has nui .'.iven students the 
math skills they need. 



The Future 

Clearly, the study t>f both the internal and external efTicient:y, effec- 
tiveness, and relevamx- of education will txcupy the attention of policy- 
makers and educational researchers in ^-ears to aime. In the nnitc»d 
States, NAEP attempts to provide increasingly cx)mplex data to help 
assess efficiency and effectiveness, and lEA attempts to do a similar ph 
internationally. Researchers are calling for greater attention to input 
variables that can make a difference and that we can do something 
about. Other researchers and educators suggest mt»re qualitative 
studies and more cautious interpretation of quantitative survey infor- 
mation. Sonw suggest greater cfK)|x*raik)n Ix-tween NAEP and lEA, but 
note the difTerenccs in ajiproach. Especially difTicult to reconcile is the 
difference in the way test items are selected. The NAEP instruments art 
constructed by panels of experts outside the schools system, while the 
lEA t&!ts are curriculum-based and designed to measure only wliat is 
taught within a system. 

The dinger, of course, is that ix)licymakers will expea prescrip- 

f\ A 



ERIC 14 



lions from such studies and may ust* tlxrm u> justify innovations or 
reforms ajmpJetely out erf kx-al cxintexi. Already calls are iK-ard for 
more htimewtiric in U S, schools sino? tnhcr cxiunirics that assign more 
homework appear to do l^etter on lEA tests But would students ck) nuw 
iKjmework in U.S. schixils, even if it rn're assigned? Are the other objec- 
tives (rf the 5ch<x)l and the many extracurricular :Kiivities in conflia 
with more homework? And even if all students were to do more honie- 
work, woukl it really make a difference? 

A rcpon jasi puHished trf a seven-yrar study of homework in U.S. 
schools and its a|pareni effeas suggests that rigid fnjni>?wt)i1t require- 
ments may have Httk? to d with increased k?arning. Homeworit must be 
related to the needs trf each student, and sk)wer students trften get s<,) far 
behind by junior high school that homemirk assignments are rK> longer 
relevant to them, so they simply stop doing them. More homeworit, this 
study suggests, could further discourage slow students to the point of 
encouraging additional school dropouts (Epstein 1988). 

Ant ^her notion arose from international comparistms thai show 
that some countries using national standardized achievement tests for 
seleaion purposes (especially for university admission) tend to do bet 
ter in lEA studies than countries asing a more decentralized seleaion 
process. If these tests encx)urage achievement, the reform diat would 
m^e the most difference at tlx* least cost wtjuld be it) use centralized 
national testing as a basis for seleaion ti> uf^r levels <rf cducaik)n. In a 
seasc, the LIS, is moving in this direaion, with a national curriculum to 
a large extent defined by textbiwk publishers and with an increasing 
emphasis on standardized testing to rate teacliers and sch<X)ls and to 
assess the ability of students to move up the educational ladder 
(Ht7nenun and Fagerlind 1988). 

The lEA studies are at the center of an evolving worldwide data 
base of tarralizing "indit^tors" of quality, effeaiveness, and efficiency. 
lEA and other groups will increasingly use e«hmjgr.^hic, case study, 
longitudinal study, and other methtxk)k)gies to complement the survey 
research done tcj date. I\)lic>makers. in turn, will select thtise "indica- 
tors" tliey see as relevant when they propt>se educatkina! reforms, 
Where all of this will lead, nobody knows, but currently fashionable 
technocratic rationality creates an insatiable demand for hard tiata to 
use in making ptjlicy decisitjns, Ix't us {M)pe we have the wisdom to 
interpret the data appropriately in our search for better scIkxjIs and bet 
ter futures for our children. 
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The Problems of 
Coiiq)ariiig Curriculuiiis 
Across ^ucational 
Systems 

lANWESTBUKY 

Figua' 2.1, a summary erf some erf the findings irf chc lEA Second 
International Mathematics Study (SIMS), illustrates the 4cind erf 
cx)mparison trf achievement th^ played a key role in recent dis- 
cussions of the quality of US. schools.* The form of the argument, now 
firmly standardized, goes like this: 

Compared with those in cHher countri^, U.S. ^udenis are under- 
achievers If our 3«:Ikx31s are to compare fevorably with the schools erf 
other nations, ;«:hievemenl must be raised. Tb ik> this, the argument 
continues, we must introduce more demanding currknalurra, such as 
thase found in more successful school systems This rhetoric is now so 
conventional that it became part of 1988 presidential eleaion cam- 
paigas. 

The argument has some n^rit. A persuasive body of theoretical 
analysis and empirical evidence suggests that curriculums are causally 
associated with achievement, and U.S. curriculums are, on their f^', 
less demanding than others in their coverage. But before the implica- 
tions that seem to follow from such findings aav tx: pushed through to 
any policy prescriptions, a number of real and very thorny problems in 



^ These findings were included in tl)c US I^panmeni di Education's 
wlctely distributed report, Whaf Wtrfc^ 

Jan Vlbsttn4r}> is Pn^essor cf Secondary^ Ecitication, r)epartnien( ofSec^ 
ondar)' Education, Unii^ty oflUitiois at Urbana Champaign, 
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Figum 2.1 

Algebra (Population A): Mean Percent Correct 
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comparing curriculunxs across sdiool systems muf^ be foccd/ T\vo of 
these issues are very important. Ihey are, one, the need to discover 
what international comparisons of achievement (rf the kind seen in Fig- 
ure 2,1 really tell us about what is happening in education;il systems 
And, two, does this have implications for policy relevant thinking about 
ti^ curriculum that could enhance U S. achie\nenient? 

I first review some cf the results (rf the aimparative analyses erf cur- 
riculums in mathem^io; that were undertaken in the course of SIMS 
(Ttavers and Westbury 1988)/^ Any claims about the saipe or depth or 
quality pf one school systems curriculum or its achievement vis-i-vis 
another must be made cautiously and re\iewed critically Paradoxically 
however, when crtxss-system comparisons are made cautiously and care- 
fully they may indicate greater pmblems than might seem ^parent 
from a quick pass over the findings 

The Seccmd International Mathematics Study 

Figure 2.1 depias achievement of one of the two cross-national 
populations investigatc^d in SIMS. Population A students were in tlie 
grade at which the modal age was 13. In the ll.S,, these are students in 
Gracte 8. In addition, SIMS investigateti the achievement of students tak- 
ing an advanccxi course in mattematics in the terminal grade c^eiicli 
system's secondary system. Papulation B in the U S. con:"istai t)f students 
taking courses for which Algebra 11 is a preitfquLsite. 

In addition to testing students on a common set of cognitive insrru* 
ments, SIMS colleacxl a lai^^e body of data on the mathematics curricu- 
lunis at the target gracfc lt*vels in tlie participating systems. To organize 



Within lEA, the terms system* or ^educational sysiem' are preferred to 
'country'* 'iialion/ *Education;il .systems" which are mi countries ur nations, 
eg., Can^lian provinces of Briiish (x3lumbia or Ontario, panicipatcxl alongside 
•systems* like the U.S. or FrarKe. The use of a fK'ulral lerm like 's\ stcm* elimi- 
nates the nerd to coastantly qualify usage 

^The following s>»siems participated in SIMS: Population A— Belgium 
(Ffemish arai FrenchX Canada (Briiish Columbia and OntarioX England and 
Wales, Finland, France, Hong Kt>ng, Hungary, Israel, japan, lAJxembouig, the 
Netherlands, New Zealand, Nigeria. Scotland. Swazibnd, Sweden, Thailand, and 
the USA; Population B— Belgium (French and Flemish), Canada (British c:i)lum- 
bia and Ontario), England and Wales, Finland, Hong Kimg, Hungary; Israel, 
J^>an, New Zealand, Sct^land, Sweden, Thailand, and the I I S. 
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ihis data and the resulting analysis. SIMS disiinguLshed two curricu- 
lums. the intended currkMhtm and the intplemetued curriculum. The 
intended curriculum was defined as the formal prescribed curriculum 
contained in national courses of study. The implemented curriculum 
was the curriculum actually taught in the schools. 

CcMnparing Cuniculums Across Countries 
and School Systems 

How can intentted and implemented curriculums be described 
cross-nationally? Within SIMS, the pools of tet; items developed by the 
study determined the framework of the description. These menas of 
possible topics for a mathematics curriculum were developed by con- 
sensus of the i«presentatives of the participating systems and contained 
tc^lcs covering a wide-rai^ing common core of mathematics as seen 
internationally TWo iasiruments provided the date. They are as follows: 

1. Using each population's item pool as a staning point, national 
committees of curriculum experts from each system were asked to rate 
each item as appropriate' or inappropriate' to their system^; intended 
curriculum. 

2. Teachers in the sampled schools in each system were asked, via 
questionnaires, whether the content underlying each item on the test 
their classes were taking had been taught to their students (the inple- 
mented curriculum> 

Analysis of these data revealed wide differences in some systems 
between the intended and the implemented curriculums. This has been 
a common finding in evaluations of many school systems. For example, 
Keitel reported that the Soviet curriculum held up to Americans as an 
example erf what our principal international antagonist was te^hing in 
mathematics— with all that might mean for both mUilary preparedness 
and technological compeiitWeness— was an intended curriculum, not 
the curriculum aaually taught in Soviet schools. 

Difietences between Intended and lnq>leraented 

CurHcuhims 

Figure 2.2 compares the U.S. and New Zealand ct^mmittees' esti- 
mates of the proponion of the items in the SIMS content areas of ele- 
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Jt^niary funaions and calculus (ft^ulaiion B) and algebra (Population 
A) they considered 'appropriate' for their systems' students with the 
reports df New Zealand and U.S. teadiers about the same items. This 
information makes an important point about the difficulties in securing 
a picture oi the curriculum of a system. The U.S. committee provided a 
much more ambitious picture of what Population B students cover in 
calculus than did their teaclwrs. In New Zealand, on the other haiKi, ihe 
two pictures are vinually the same. However, for junior h^ school 
afeebra, the New Zealand nmional committ^ <^ered a very different 
ptoture of the curriculum than did cJa^room tejcheis In contra«, the 
US. committee and US. classroom tethers presented vinually the same 
piaure for algebra. SIMS evaluated the jiKlf^nents crfboth national com- 
mittees and teachers against what st uctents actually learned as measured 
by the study's cognith^e test and discovered that, in ahnosi ail cases, ibe 
deso-^atons cf Ox in^emerued curriculum fsmnded by tmcfjers umv 
a better pmMat^ adi^vement than the apprcprkuene&s ratings pro- 
vided by national commOtees, 

How could such comminees so misjudge what was happening in 
their school? And why, as we have seen, were these committees .so 
inconsistent in their judgments (or misju(j^.ents) acrwss content areas 
and populations? After all, each SLMS national committee coasisted of 
"true" experts. In New Zealand, they were inspectors of mathematics 
education, whose job was to visit scIkjoIs and assess curriculums and 
teaching, and experienced tethers. In the US., they were mathematics 
teachers, supervisors, and textbook authors. 

In the U.S. national committee's ttescription of the calculus course, 
we know the answer to these queaions. The committee focused its 
Mtentfon not on Grade 12 mathematics classes seen overall (i.e., the cur- 
riculum of Population B classes), but on a subset of these classes, the 20 
percent of Gi^de 12 classes where a formal course in calculus is taught 
By implicaUon, the committee unconsciously projected its sense of what 
should be taught in Grade 12 to the whole population. They did not 
make this mistake in the case of Grade 8 algebra. In New Zealand, we 
know that pan of the nattonal committee's prc^em stemmed from the 
range in levels of coverage of algebra that occurs in New Zealand's 
Grade 8 classrooms. Despite the presence of one formally prescribed 
national curriculum, significant de facto tracking occurs in New Zealand 
schools, which results in highly differentiated curriculums and highly 
variable achievement, The New Zealand committee has acknowledged 
that is members did not ^predate the extent of this tracking until they 
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RepOTled Coverage 



saw both teachers reports of their implemented curriculums and the 
variation in achievement found in New 2^and's Population A classes 

Many SIMS national committees experienced similar proWems in 
rating the appropriateness' of the SIMS item pools to their curriculums. 
As my examples illustrate, such judgments about the scope of an 
intended curriculum mast always be evaluated in the light of qualifica 
tion for understanding that might (or might not) be clear to insiders 
firom a panicular system, but will be unclear and potentially very con- 
fusing to outsiders. And, as I have shown, one of these qualifications is 
withln-system variation in course coverage. 
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DescriWng Variations in the Distribution of the 
In^emented Carricuhim 

We have seen that the pktures erf the implemented curriculum that 
came from teadwrs may or may not match descriptions of the intended 
curriculum based on expen testimony. But in addition to this validating 
role of teacher-based descriptioas, we can also use teacher-provided 
data to secure a piaure of within-system variation in curriculum. 

Wiihin-sysiem variation and diversity are key parameters of any 
national curriculum and, as we have seen, Uiey are parameters which 
can modulate in significant wa>'s descriptions of a curriculum that 
focuses on modal panerns alone. Such panem- of distribution of 
"opponunity to learn" arc, in their turn, related to achievement in very 
imponani ways. 

Figure 2.3 on page 24, for example, describes the variation in cov- 
erage of algebra in a subset of systems. In these systems, teachers of 
high-coverage clas.ses (defined as classes at the thirxi quanile, i.e., Q3. of 
each system's distribution of teacher-reponed "opportunity to learn" in 
algebra) indicated thai their students had covered more than 85 verran 
of ilie content in the SIMS algebra item pool. The level of covc^e in 
these classes is represented by tl^ top of each line in the figure. As 
would be expeaed, little variation appears across these systems in the 
algebra taught in their highen-coven^ classes. But a quite different 
pattern emerges when we look at the level in low-coverage classes 
(defined as classes at the first quartile; i.e.. Ql, of each systems di.siribu- 
tlon of teacher-reponed "opportunity to learn"). In Japan, low-coverage 
classes experience a curriculum that is virtually the same as the curricu- 
lum of the highest<overage classes. Substantial equity in coverage 
exists across all classrooms in the system. In the U.S., on the other hand, 
curriculums are extremely diverse, and a substantial dilTerence in the 
levels of coverage exists for both high- and lo^<overage classes. lJ.*i 
teachers erf high-coverage classes report teaching about 90 percent of 
the content represented by the SIMS algebra pool, whereas teachers of 
iow-coverage classes rejwrt tea. hing only about 55 percent. 

Figure 2.4 on page 25, taken from SIMS U.S repon (McKnight et al. 
1987X suggests what this variation in coverage implies for the patterns 
of achievement of VS. students when compared with the patterns of 
some other systems. It is an outcome of the between-class tracking 
omnipresent in Grade 8 classrooms. But when Figure 2.4 is viewed in 
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the context of Figure 2.3, wo important questions arise. Why is the pat- 
tern of curriculum offerings in the U.S. GracJe 8 so much more diverse 
aiid inequitable than the pattern seen in other systems? And more 
important for our theme, once we recognize this inequity, on which 
dasses or class types should wc focus policy prescriptions to improve 
the curriculum? The curriculum offered to our highest-coverage (and 
highest achievement) cbsse.s is not a core problem when seen cross- 
culturally, but the curriculum offered to our lower (not lowest) cxjver- 
age and achievement classes is. This answer docs not emerge from a 
focus on mean or modal, intended or implemented coverage data alone. 
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Rgure 2.3 
Algebra: Coverage at 03 and 01 for 
High-Coverage (03) Systems 
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Population A: Achtevament VSarianco ^ Rvo SIMS Systems 





TTwse drctes d^fct for eiwh country the variance compcwienis associated wth tf^ 
core pretest, (For most ccx^tries. this test cOTSists of 40 items For Japan, it had 60 
items ) The aieas the circles are roughly pfoportional to the total amwnt of score 
variation. The slices represent tt^ amoflit of tc^al variatbn that is attritwtable to differ- 
ences between students, classrooms and BCfwAs. Since these are pretest data the 
variance cxxnpor^nls represent how students were albcated to schorts and class- 
rooms and nc^ to differences in lea:hing during the school year. Very similar patterns 
were found in the posttest dma, as well. 

^jufce Curtis C McKmghl m al (1987) The Underachmff^ Cumadfurn Assessmg US Schoc^ 
MsthBmalfcs from an fntemaikyia^ f^$rsp^tfVB. Champaign, ill Stapes. !9B7. p tOB 
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Hie Yields of Mathematics Educatkm 

Who and What Should We Evaluate? 

Within SIMS, the curricular and achievement quesaions that circle 
around the ultimate outcomes of mathematics education in each sdKxd 
system were posed in terms of two L«5sues: How much mathematics do 
ftjpulation B s adents team, and how many stuctents are given the 
qjportunity to learn that mathematics? We used the icrmyield to gather 
these questions within one frame. How much do systems vary in the 
yield erf mathemMics education? 

Let us consider these questloas hy focusing on one course. Hew 
many students in the various systems are »ven how much oj^rtunity 
to leam calculus and how miKh calculus do they learn? Figure 23, when 
viewed beside Figures 2.6 and 2.7 (ppt 28-29X gives us the starting point 
for considering what w have learned atwut this from SIMS. 

From the U.S. point erf view, the findings implicit in the figures in 
these cross-system comparisons are, at first glanc-e, disconcerting. U.S. 
arhievemem is low; the numbers taught in Grade 12 are not paniculariy 
high; and, when we view achievement and enrollment rates together, 
US. yield is among the lowest found in the study. 

But three issues need to be considered as we try to assess these 
findings and consider what they might mean for U.S. curricular policy 
making. First, the questirm arises of the meaningfulness ami validity of 
using grade VZasthe point of reference for a cross-cultural considera- 
tion of yield. Second, who do we count in the pool of potential enroll- 
ees in a calculus course and, therefore, what tknoniinator do we use to 
develop an index of an enrollment rate or retentivity? Finally, what is 
the significance of calculus itself as a criterion for assessing yield? These 
questions are difficult to answer. 

Who and What Should We CcMii|»Te? 

In Europe and in systems such as New Zealand or Hong Kong, 
which have highly selective struaures at the secondary school-a>llege 
interfece, the curriculum of the acalemic secondary school is tightly 
linked to the university and, typically, university programs have been so 
framed that students cannot pursue studies in most subjects without tak- 
ing the highest level courses offered in their secondary schools In 
mathematics, physical science, and engineering, for most of this cen- 
tury, that course has been calculus This has not been, and is not, the U.S. 
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Etementary Functions and Caicuius: IMean Percent Correct 
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Figure 2 6 

Percent of Age Cohort Retained In Advanced Mathematics 
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patlcrn. In the IIS, a highly pernieabk* boundar>' exists betvmra the 
schiX)! and the ailJege or university; conseijuently, colleges offer many 
courses thirt Europeans wt^uld regard as quiie inappropriate. Further- 
more, in tlie LIS. mathematics curriculum, iniroduaory^ calculus has nut 
been a schix>l course. It has been a university amrse and, until the post- 
war yews, taken in the sophomt>re year by most students. Only since the 
late 1940s and early 1950s has even the idea tliat c^alculus should be 
taught in .schm>ls been seri<:)usly entenained. The reality^ of sclux>l 
courses in calculus is cvvn more recent, and today such courses are not 
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so widel>' taught that aillcge programs can presume thai students have 
taken more than precalculi'.s. Moreover, the kxx^e overall framing of 
cdkfge programs meaas that students may enroH in mathematics 
courses with only, say, Algebra 11 in their sdicx^l programs, Tliis implies 
that students need not take, nor schtxils offer, more than a senior-level 
precalculus course. 

Given such fundamental historical and structural differcmx-s 
between European and ILS curriculums, what nwaning can be ascribed 
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The Yields of Mathematics Education In SIMS Systems 
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to crass-cultural comparisons of curriculums and ^iiievemem? IIS, 
schools are not cx{Mxted to ^^er courses in calculus, even to students 
heading towards mathematics-txised colk^ programs, and many 
schools do not. Wouldn't it have been more meanii^ul, An^ricans 
might ask a study like SIMS, to compare oppc:>rtunity to learn and 
adiievement after the point at which the relevant aihom had had real 
op^nunities to fearn calculus? Funhernwre, given the requirements 
for mathematics courses found in most US. colleges, could we 
expea U.S. yields of mathematics course-iaking and achievement u> 
took quite different for students at age 22 than ar age 18? In c^her words, 
j^ruaural and cultural faaors inevitahl>' come into play in very central 
ways as one consitkrs what the questions and the target populations 
might be in any cross-cultural siudit^s involving the LI S. and in inter- 
preting the finds of such studies, let me illustrate this observation in 
amHher way. How many are taught and learn mathematics in the termi- 
nal secondary' grade? 

One criterion for evaluating the success of a subject within a schcx)! 
system is student nent, Tlius, son>e yt-ars ago we often heard, for 
example, that in ;0 percent or more of the siuifcnts enrc^k*d in 
advanoxl matheniaikrs a^urses in that system's gr^* 12. That figure was 
often compared to the ^^iparently much knvcr enrollments by U.S. stu- 
dents in upi^r level schwl mathematics courses — with all the dire 
implicatkuus that seeirK?d to folkw for the future of our hi^^ technolc^ 
economy. And that brings iw to a discussion of tfn! narrower question: 
How many stuctents are taught mattematics in i\w different schcxjl sys- 
tems that panicipaied in SIMS? 

The most striking finding to emerge from SIMS was the wide dif- 
fereiKes in the numlx'rs of studt^nis enrolled in the terminal grade in 
sch(x>l We faced considerable difficulty in coming up with appropriate 
estimates of this seemingly simple statistic {Westbur>^ 1^). In the U.S. 
at the time of u *e jaudy. approximately 85 percent erf the ^ €ci\on were 
enrolled in the 12th gracfc in an undifferentiated high schcx)l system. In 
Japan, 92 percent were enrolled in the equivalent of Grade 12 but, at 
this level, the Japanese schcx:?! system is highly differentiated into scho^>l 
and program t>pes, In England, on the other liand, :^)pr^jximately 18 
percent of the age cohort were enrolled full time in the sixth form in 
sea>ndary sch(X)ls, preparing for advances! level examinations — but 
anotharr 12 percent tocik advancaJ level courses in coIlq?es of further 
education. Yet another 20 percent or so toc^ full- and pan-time occupa- 
tional or professional training outside the academic system. 
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Such statistics iJlustrate the aimplexity btHh {>f developing a nor- 
mative .sense of what sbm/d happen in the last y^r di high scIkx>! an 
iastitution that takes cliffere.it forms in different systems. We must ask 
then, who is in the high school? What part of the age cohort do these 
students represent? What kinds of murses arc taught in high schtK)ls 
and why? Only then can we attempt to derive meaningful contpamtnv 
estimates t>f how many students are enrolled in any course at the termi- 
nal level of secondary schooling (Westhury 1988). In, for exampkf, the 
statistic for Japan— that 30 percent of students enroll in advanced math- 
ematics courses— the question is which students make up the denomi- 
nator in this percentaj^. On further invesii^ion, it turned out that the 
denominator was made up of only students in academic programs and 
schools. When all students in sch<X)l were used as the denominator, the 
retentivity index became 12-13 percent— effeaively the same as that of 
the U.S. 

Because of these kinds of interpretive and technical issues. SIMS 
sought to discover Ikjw many students ^-ere enrtdled in advanced math- 
ematics as each system defines it by using the age a)hort (all >t)ung ptx)- 
ple of the modal ^e of Population B students in the <:ountry) as the con- 
stant base for all comparisons. Figure 2.6 .shows SIMS' estimates of 
enrollments in Population B mathematics derived in this way, and Fig- 
ure 2.7 relates these findings to achievement on the SIMS cognitive 
tests. As we see, substantial between-sy^fem differences ^pear in the 
numbers of students who persist with mathematics to the advanced 
courses of the terminal year. (The U.S. stands in the middle of the range 
wirh an enrollment rate of 13 percent.) And. as seen in Figure 2.7. lower 
enrollment rates are generally associated with higlier levels of achieve- 
ment and vice versa. However, some of the lower achievement .systems 
in the top right quadrant have very high enrollment rates, and .some s>'s- 
tems achieve both high enrollment rates atiti high achievenK'nt. 

The between-system differences seen in Figure 2.7 also highlight 
the range of policy goals that might be available to national educational 
systems in any subject and, by implication at least, imite us to ask ques- 
tions ihat are ftmdamental for both assessment erf these differences and 
policy nrescriptk)as that might folltw: What kinds of goals for enrt)ll- 
ment and retentivity in advanced mathematics arc mast desirable' Is it 
always desirable to emphasize high achievement as a goal at the 
expense of participation? Is it self-evident, for example, that schix>l sys- 
tems should choose policies that defer to achievement at the expense of 
high levels <rf enrollment? Is it self-evident thn achi^rvement in any sin- 
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gle subjea or area such as calculus should be tfx' criterion to invote as 
we coasider how to rank or rate any educatitmal system vls-a-vis others? 

TTie importance o( the last question is highlighted for iNonh Ameri- 
cans as we consider the panern of yield seen in the SIMS' findings fixjm 
British a>lumbia (see Figure 2.7): 30 percent of the province's 18-year- 
olds take the system's terminal level advanced mathematics ct)ui^. 
Although (as seen in Figure 2.5) these students' achievement is low in 
cakulus, it is middling in algebra. How should we evaluate such a pat- 
tern of yield? In tlw words of McLean, Wolfe, and Wahlstrom ( i9H7), 
writing from the perspeaive of Ontario: 

One hears complaints (in Ontario] that entering university stu- 
dents are ika adojuaiely prepared in mathematics, that iliey need 
to relearn what they have studied in high school. A particular 
issue in mathematics is the utility and efficacy of tlw iastruaioi' in 
calculus. In Ontario, virtually all mathematics specialists take cal 
cuius in high school This is aim true of mast other countries, 
with the notable exceptions jrf the USA and British Qjlumbia. In 
British Columbia, calculus is simply not available to students, and 
the senior mathematics program is a rathei thorough course in 
algebra and trigonometr>'. Tlie universities expea to tt^h calcu- 
lus, . . . Ontario students study calculus in grade IZ and then study 
it again in first-year university. Their {xsrftjrmance was fxx>r on the 
SIMS trigonometry item subset. 

/s it realfy c^)pnjpruue m fMtt' so much enipbasL^ on cakulus in a 
-rvsrew (like Ontario} that Ixis siicb a frigh participation rate { 18% 
of the export}? . . . time could he better spent in tfje Ngb sdxHjk 
conrolidating pre-caktUus mathematics or on other topics alto 
gether. (Emphasis added) 

In Ontario, such reasoning provided a rationale for a new Grade 13 
course in finite mathematics, covering matrices, combinatorics, proba- 
bility, and statistics. Many mathematics educators in the U.S. and else- 
wliere would agree that such a course, with the implicatitjns it might 
have ft)r mathematics for all, represents the direaion in which thinking 
about nodern curriculum in ad\'anced schot^l mathemati« is going— 
this is in contrast to the selective assumptions of the late 19th-early 
20th century curriculum that we have inherited, in which calculus Is its 
culmination (se-? Damerow and Westbury 1985, 1987). 
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As we consider the issues raised by McLean, Wolfe, and Wahlsirtim 
(1987) in the context of this paper, we again have to ask what they might 
mean for the ways in which we interpret the impHcations of the SIMS 
findings cm US. yield in advanced mathematics. W can still be con- 
cerned about the position of the U.S. in Figure 2.7, but. as we consider 
what should follow from this concern, we mas? wander whciher we 
should devetop curriculums which tbrast, on the one hand, in the direc- 
tion suggested by the course revisions in Ontario or, on the other hand, 
in the direaion suggested by low achievement-high enrollment .sys- 
tems such as British Columbia. In other words, do the findings of SIMS 
suggest that we should retool our conceptions of what should be taught 
and how many students we might try to reach in the light of our educa- 
tional structures and assumptions and aintemporary thinking? And, of 
course, we must wnstantly question what such thinking means for the 
ways in which we interpret the results of studies like SIMS. 



In this account of the problems of interpreting SIMS results, I have 
highlighted the difficulties we face in deciding what the findings mean. 
All tliinking about schooling feces the problem of escaping the limita- 
tions we share as a result of our experience in one school .system By 
necessity, involvement at any level in a cross system studv foires us to 
stand outside our own framewi>rit, for a tinv at least. The inevitable 
result is that we see our worlds differently Although I have empliasized 
the problems that must be faced in interpreting the results of a study 
such as SIMS, I wouW insist that these problems are not a weakness of 
the study but, rather, its strength. By the coasidering questions I have 
raised here, we secure new understaixlings of the limitations of our 
often unexamined assumptions about what schools are, what thev 
what they achieve, and, mo.st important, what they are striving to 
achieve. 
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How 1^0 Educational 
Systems Learned firom 
Conqiarative Studies: 
The Swedisli Ej^erience 

INGER MARKLLJND 

To clarify the impact of lEA's moltinational and other interna- 
tional and national rescanrh on Swedish educational policy, 1 
must finit describe several charaaeristic features of Swedish 
society that influence the political impaa of research. Some of these 
may, indeed, be international, while others are quite distinctively Swed- 
ish. Of the following five feaors, three are definitely Swedish; the other 
two probably apply to many countries. 

Reform Strategies in the Educational Sector 

Important changes in Swedish stxrial policies have, until the past 
very few years, invariably been the result of studies prepared and pro 
pased by large government commissions This is eminently true of our 
educational reforms since the 1940s. These commissions often include 
researchers; indeed, are sometimes chnired by them. In addition, 
researchers arc engaged as expen advisei^, research secretariats are set 
up, and rest-arch is generally undertaken at the request of government 
commissions. Research and researchers are pou^rful determinants for 
change in Sweden. 



Ing^ Afarklund is ReseanJj Direcu,r Defxirtment (tfCoordimuion and 
Pkmmt^ National Board cf Education, Stockholm, Suvden. 
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The System of Central Administrative Boards 

In Sweden (and FinlandX \hc administrative, executive part of 
national government a>asists of separate, independent central ^Iminis- 
trative boards which, formally, are directly responsible to the govern- 
ment and not subordinate to any individual ministry Tl^ national 
Board of Education (NBE) is one such authority. The boundaries 
between designing znd implementing a policy are not immutably fixed. 
The authority that Implements political decisions also interprets them, 
which gives it the acklitional role erf policy maker As far as education is 
concerned, this v,^ to a git^at extent true during a tryout period in the 
1950s, before the nine-year compulsory, compreheri:;ive school was 
decided upon from 1962 onwards. In recent years, lR>wever, great 
emphasis has been put on a clearer distinaion between tlie political 
and administrative decision making, the former resting with the Minis 
try of Education and government, the latter with NBE. 



Swedish Research Fcdicy 

Research in Sweden in the liberal arts and stxrial sciences is con- 
duaed almixst exclusively within our universities Th'?y receive grants 
from four sourcc^s: the state, rt^search councils, national authorities, and 
industry Of the four sources, the grants from national authorities are 
most important for education. 

The great majority of national administrative bi^ards, such as NBE, 
receive specia* research alk>caiioas and are required by their ioanding 
instruaioas to initiate and finance research. This form of funding is 
referred to as 'The Seaorial Principle" NBE has received a special 
research allocation since the compreheasive compulsory school was 
intrcxiuced in 1962. Its standing instructions require it to plan research, 
to define research priorities, and ro specify finance, atid utilize research 
as part of its responsibility for school devek)pment. This duty distin- 
guishes this seaorial ly-funded research from the aaivities financed, for 
example, ihrougli research councils. The national authorities, Ixwever, 
are normally not required to engage directly in research. Thus we do 
not have an; in-house research aaivities nor do we have separate 
research institutes for our educational research. 
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These three characteristics are ih«se distinctively Swedish. Tlie fol- 
lowing prtibably apply to many CDuntries. 



Intematicmal Cooperation 

In the educational seaor, four overlapping agencies are important 
in Sweden, though educational questions account for only pan trf their 
aaivities. These agencies are the Organization for Economic Coopera- 
tion and Development (OECD). the a)uncil of Europe, UNESCO, and 
the Nordic Council. Of these, OECD unquestionably means most and 
has the greatest impaa. To a great extent, OECD constantly refuels the 
Swedish sensitivity concerning our position in various international 
ranking orders. 

Finally, a word about the impaa t)f r^earch on politics, policy, and 
practice and the conditions favoring or impeding this kind of impaa. 
One can distinguish bcah the nature and the degree of different effeas 
of research, as well as correlates of the effeas. In faa, a governmental 
committee that reviewed NBEs research and dcvek)pnient work 
between 1978 and 1980 employed this distinaion. Incidentally, that 
committee was chaired by a professor of education, another <)f its mem- 
ben; was a profe.ss<.>r, and two of its spet ial expen advisers were profes- 
sors. In all. about 20 percent of the pntfessorial corps in education at 
that time was that particular cxmmittee. 



Efiiects and ££fect Goirelates 

Effects can vary in kind from general to sixxific, from attracting a 
certain amount of interest to aitraaing interest in a very s}>f.*cific aspea, 
say a new teaching aid. Effeas can also vary in degree from no efftrt at 
all to either a very strong positive or negative effea. 

rffects correlates, that is, what helps create effects or ^ack of them, 
are related to both kind and degrce. Usually, only positive effects are 
kx^ed for, but equally important is looking for what "helps'" create neg- 
ative effects. 

Because Swedish charaaer.stics make it easier for research and 
research findings to get aaoss to politicians and other decision makers, 
researchers and research are strongly represented in the process of 

o — ^ 
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educational reforr' and in our system of pc^liQ^-i^xecutive and policy- 
making txxiies;; research and research utiiizaiion are impi^nani fields of 
respoasibility. This ^ds up to a receptiveness of research and its find- 
ings. Include in this equation our dependence on the outside world and 



Figure 3.1 
Kind of Effects 

Oneral 
effects 

Attract interest, create debate 
Infitance attitudes 

Give rise to new organiyatkxial models 
Give nse lo administrative rules and routines 
Influence curriculum 
introduce new instructicnal methods 
Create new teaching aids 

Specific 
effects 



DEGREE OF EFFECTS 

strcng no strong 

negarve effects positive 



Figure 3.2 
Effects Correlates 

to kmd of effects 

• problem area (broad, narrow, general, sf^citic) 

• target group (ger>eral public, well-defined, small interest group) 

to degree of e/fecf s 

• the distinctiveness of the message 

• information processes 

• tf^ availability of efficient channels of communication 

• the readiness to receive the results of research 

• the increase of knowledge, bdh practical guidance and theoretjcal insight 
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our need fi>r an economic and cuUural phne wiihin tin? international 
community, and >t^u H principal mechanisms explaining why 

multinational research, yixwc all within the lEA, and genei^l interna- 
tional research have had an impiKi on our educational policy and will 
probably continue to di^ so. The impaa on educational praaio? has, at 
times, been a good deal less or, perhaps more accurately, it lias Ixvn less 
direa. 



ffiA ^iKlies: First Generation 

My first example is the first generation of lEA studies— mathemat- 
ics and the six subject survey The first mathematics survey (I-IMS) was 
a^nducted in S^neden in 1964 and refX)rted in 1967. It ought to have 
aroused greater ait**ntion than it aaually did, considering what hap- 
pcntxi when tlie second matliematics survey (SIMS) was rejx>rtcd 
between 1 983 and 1 985. 

The results of the six-subject survey were published in early 
autumn of 1973. Ever since 1970. a fierce pK)litic:al debate Iiad raged in 
Sweden about compulsory schooling and tte decline in standards of 
achievement which, it was alleged, resulted from sul>slituting this 
school system for the system of parallel schtx)Ls, Four major issues ikcu- 
pied the focus of attention: marks, iiiscipHne, foreign language teaching, 
and the rising proponion of sjx^cial teachinf 

This debate fell silent in 1973. Sweden's position on the interna- 
tional score sheet belied the great majority of ad\x'rse pronouncements 
on Swc\iish schtxrfing — in an international light, be it noted. 

From 1973 onwards, lEA has been refer rc\i to freciuently in the 
Swedish dtHbate on educational policy IRA proved a source of Outstand- 
ingly durable arguments in favor erf the a^mprehensive compulsor\^ 
schcK)!. ft*rhaps the most important finding of all concerned the 
berween-sckx>l variance in Swetfcn compared with other countries. To 
those of us who hav^e made equivalent educational stanAirds a corner- 
stone of our cxlucational {X)licy lEA furnished a)nvincing evidence that, 
in this particular respect at least, our t\iucational reforms had up to then 
been successful. 

Using the categorization of eiTects and effect a)rrelates, wc get the 
following piaure in Figure 3 3: 

/ (\ 

Er|c 39 



The Svp-EWSH Experienck 



Figure 3.3 
First-Generation lEA Studies 
(mattomatics and sIx-sutH^ survey) 



EFFECTS 



General: 



Created debate in educational politics and policy about educa- 
tknal standards ar>d evaluation 



Specific: 
0 



Nk> effects 



EFFECT CORRELATES 



To the 

ger^ral 

effects 



Target groups (polltiCfsns). lEA results published during a heated 
political debate about standards in the comprehensive schod 



To the 

specific 

effects: 



The distinctfveness of the rr^ssage; lEA variables and data to 
general as compared to national curr icuium 



TEA Studies: Second Generation 



A major rammissk)!! to prepare for a reform of teacher education 
progi^s was already appointed in 1974. but not until 1985 was a gov- 
ernment bill on the sul^ea ininxluced in the Riksdag (parUament), 
Once again lEA became very much part of the picture, but this time its 
fx^ition was completely different from 10 years earlier, Th^ Swedish 
results in the second mathematics survey (SIMS) had been published in 
1983, but little attention had been paid at the time. No appreciable 
chan^, for better or worse, were apparent. By now, the finding that 
Swedish compuLscrr)' school pupils had not done ver>^ well in mathemat-^ 
ics in tlie 1964 survey had been forgotten. Only when the international 
averages "leaked out'' in 1%5 did the sutvey become "hot ^uff." In it. 
the ci^ponents of the proposed reform of tether education found such 
TOnviiKing evidence erf the failure trf the Swedish compuLsc^ry compre- 
hensive scIkx)! that the tlien Minister of Education appointed a special 
study group to investigate u^jy Sweden was doing so badly TTiat our 
upper secontlary school students wx>re doing quite well remained 
unnoticed. 
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Using tlie same categories of efTecls and effea correlates we obtain 
a somewhat different piauir from the one resulting from iIk- six- 
subjea survey. 

Here we can see that, this time» the lEA has als<i had sjxvific effcvts, 
and I think these will also beanne appaa*ni in education practice. Tlie 
propasaLs put forward by tlie matheniatics group were excejMionally 
well received, not only by politicians but also by teachers as well TlK>se 
proposals are now being put into practice, and all sdxK>ls and teacliers 
are affected, Also in SIMS, the between schcx)! variance was ver\^ low, 
which a>uld be interpreted as showing that the standard of mailHfmaiics 
education was equally low all over the a>untr>'. But even in this context, 
the law between-school variance was ix>ih politically and praitically 
helpful. The measures taken in the resulting national development pn)- 
gram, administered by NBE, concern tUi sch<x)ls, all students. Without 
the lEA survey, mathematics teaching w\)uld probably have remaintxl 
fundamentally unsatisfactory for several wars longer Us unsatisfactory 
state was apparent bi-fore 19H3; the lEA findings and the international 



Figure 3.4 
The Second iEA Mathematics Survey 



EFFECTS 



General 



Created heated debate in Parliament governrT>ent. mass medfa. 
schoc^s. and anx>ng parents 



Specific 



Governmental committee on mathematics education resulted \n a 
study day m all schools, compulsory tor all teachers in grades 1 -6 
and all mathematics teachers m grades 7-9. earmarked 
resources dunnp four veafs for INISFT and development work arxl 
curriculum changes 



EFFECT CORRELAFES 



To the 

general 

effects: 



A specif'C problem area mathematics, targpf tjrajps oppo 
ner^s ot teacher educaton reform 



To the 

specific 

effects: 



The distinctiveness of the message the »nternatfonai low scores. 
efficmnt ^ hannels very alert ffied»a. read^r^ess to receive results. 
teachers, teacher educators 
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ranking df Smxlish stmienLs, in particular, were an unpleasaiii, but cvi- 
ckntly necessary, reminder 

Secxind Science Study 

The main Swcdwh findings for the sea>nd sciernx' sunx*>' (SISS) 
wre reported in August 1986. From a itti/iwio/ viei^poini, ii stimulated 
a number erf impt:)rtant, positive ro^ults, but until March 1%8, all politi 
cal, media, or other impact was aiaspicuous by its absence, What were 
people waiting for? Answer; The international re?ailts. ^Mter two years 
^nt living in the wrecks k?ft by the mathematics survey, I sincrrely 
ht^xxl that the pi:wiitive signals we had received from the Swedish analy- 
ses woukl not be swan^^ed by nasiy^ surprises from the international 
comparisoas. 1 could not help asking myself, though, what, if anything, 
wouki have happened if the science survey had been a compk?tely 
national study, if it had not been known that international results were 
also expeaed? Would the sileno? have been the san^? 

The full national report of SISS was publishtxi on March 1, 1988, 
together with ttie main international results All i*^ -^U, the general 
imprc-ssions from the first actional publication in Augu.n 1986 remained 
posiUve, though now; for some parts at least, affirmed and strength- 
ened by the international results. The a)mpulsory comprehK:nsive 
school was judged "approved*' b> the international ranking ordt r Not 
so, tlx^ugh, in science in the final year of upper-secondary school (pre- 
universityX where Swedish students scored below average on biolr^, 
chemistry, and physics. 

But this time, we could discuss tlK? international and national 
results simultaneoasly, and this contributed much to the, on the whole, 
balanced discassion thai t<x)k pl^e. TTie sht)rta)mings of the scienct? 
students a^uld be discussed in terms of what and wIk> they were com- 
pared with and this staned a discussion of among other things, tint pnxs 
and cons of a highly specialized science education, such as ilH)se of 
England and Hong Kc^ng, versus tlK* almost encyclo|Kdic upper-second- 
ary education in Sweden, 

It is t(X> early to predict the effects of the SISS study in Swedish sci- 
ence education. National and, alwve all, imernational results are still to 
a)me. The most important lc*sson to be learned from SISS is this. It is 
impHfraiive that naikinal and international results are published and dis- 
cussed simultaneously If not, internatk>nal results will remain an Olym- 
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pic game with very limikd possibilities of hciiiK cxpl;iiiK\l In jnd 
adikxi io ihc undersiundin'i of natiunal icsulis 

^VMtten Ccmiposlilon 

m- aa' now working* on thv third lEA siiid>', VC'riticn (:oni}>osnjoii, 
and neiihcr tlx* national fK)i iniernaiional anah ses haw yci Ixx'n coni- 
plctttl. 

This study is hrisicalh* different from its prttlecessors, tfie extensive 
six^jx* of ilK- rjualitativc data is siving us the pupils picture of pan of 
their reaht>' btKh in and tniiside of scIkkI ninnigh tlKMi own prcHiu' 
Uon yxv will gain a picture i>f Swxtiish nioifier tongue instruaion that no 
conventional te.^is or questionnaires could give us. 1 Ix'lieve that ibis 
muliinatii)nal study will have Ci>nsideial inipaa on t-ducational prac 
lice Whellx^r or not it will fiave any f^^ilitical imp;KC remains tt) }>e stvn 
IVriiaps wv am l<x>k forward fo Nntom up efitxis R-rhajys this time the 
changes in teaching which may result from Ixnh the national and inter 
national analysers can be djrc\lly iniiialeil by the [^"pils 

Overall Conclusions 

I would like to condutle with a fi^w reflections In Sut-den. we can 
obser\'e a tlevelopnjent of tlje impact of IVAs multinational resi-arch on 
educational fX)Jia and practice. TJk- lEA studit^ of the imh: and Wlih 
had a great dc-al of political impad, but did not m;ike an>- praaieal 
impression They did not lead toctir ricular ckinges or chatiges m leac h- 
ing methcxls, teacher eilucation, inser\ ice training, teaching materials, 
or ultimateK; of course, in the pupils' command of maiherp itics. 

The science survey can have lx>th a f^oliiical atid a praciical impact, 
and J hope and belie\e that written c^)m{X)siiion will also have an 
inipaci on tc*aching practice 

Htnv have these dt*\X'lopnK'nts come alx)ut? 

The lEA studies have panh' changcxl character AspiraiioriS have 
rist*n, and. within tl)e lEA. anahiic^al rmniels are being develo}xxl that 
emphasize systematic understanding and the truly comparati^-e asixvt 
beyond the !nternation;U top of the ftirm * 

SwcnJen has intensified its national follow up We are ?k> longer 
content with international averages and ranking orders, we are trying, 
more and more, to penetrate the backgn^und of thc^se data. We are not 

I ' - 
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gS^ing to have mathematics teaching which adheres to a kind of interna- 
tiunal core curriculum, ^EA !ias not been effective in this wav; but lEA 
fotml us to take a long, hard IcKik at wliat is fi^pcning to the teaching 
of mathematics in Svwxlen. 

In total figures, lEA is attracting more attention icxiay and from 
more pec^le. The interest shown by the media, admittedly, is oat always 
salutary. Here we have to pk?^ ^inst the self-assumed durj' t>f journal- 
ists to highlight the Imi news. IVrh;qis this dismal image can, to some 
extent, be oRsei by a desire on the pan trf politicians to see a positive 
outa^me from the ctecisioas they make. 

Finally, I would like to quote what the Direaor General of NBE» 
Erlatid Ringborg, had to say on this subject in his ^^Jdress at the 26ih lEA 
General Assembly, Open Day Seminar in Stockholm: 

Critics of the lEA sometimes bewail the tendency of politicians lo 
latch onto superficial cumparlsoas between countries as a means 
of scoring politick points, Personally, 1 feel that we shoufd refrain 
from this kind of moralizing and acknowledge the realities of pol- 
itics, If the sur\'eys have the scientific substance we assume, then 
let as inste*»d turn the whole thing to the best jx^ssible account. 
Let us ^.upply our politicians with national averages and ranking 
orders and also with ttwse analyses of tlK* underlying syslemaiic 
faaoni that may ser\e to explam the results. 
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How 1^0 Educational 
Systems Learned from 
Conqiarative Studies: 
The Hungarian 
Experience 

zoitAn bAthorv 

Ctintinuous measurement of th; effeaiveness t>f public edi«ca- 
tion has been rt^arded as a primary concern trf Hungarian 
educational rcscarchen; for almost 20 years now. By "effeaive- 
ness," we mean learning achievements rather than the quantity crfmarics 
or cenificates. We intend that the accountability of an educational sys- 
tem can justifiably be described In terms of learning achievements. 

We measure this effeaiveness in tJie frameworks dltuakmal and 
intemmionaJ stm>eys. To do ttiaf, two important methodological prob- 
lems must be solved: which gmtJa^ and which fields of education 
(scImkA subjects) do we survey? 

In both Hungarian and international surveys, we usually :^point 
three .strategically important grades: Grade 4 as the final >'ear of elemen- 
tary school education (Population 1). Grade 8 as the final year of the 
eight-grade general school (Ropulailon 2), and Grade 12 as the school- 
leaving year of secondary school education (ftipulaUon 3). 

The selection of fields of educatioti to be surveyed also needs care- 
ful consideration. In Hungary, the view is fairly common that the fields 
of eduG^^ion chosen should be hnrh ftjndamental in the learning pro- 
cess ana those in which a high degree of validity can be ensured. In our 
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qiinion, reining cc^mpreheasum, rriathemaiical-k>Rical skills, and the 
o^itivc side of science eduation can satisfy these two criteria. 

We are, of course, aware that this system provide, a rather rcxluced 
view of the evaluation of school education. We know that the effective- 
ness <rf formal education cannot he reduced to some basic skills and 
knowledge (minimal competence). Nevenheless, in order to adiieve 
"sdter'* educational effects in the develo'-ment of stutkuLs pcrsonali 
tlei, a basic standard erf fundamental skills and knowledge is needed as a 
reliaWe sysrem trf guarantee. Our surveys, therefore, intend to inform 
teachers, sdKKA admini^rators, and rese-urhers if this basic stan'iaru 
exists for the sdter, more complex effects. Therefore, v;e ransc -ously 
^cpt the reduaionism <rf our ^itosophy evaluation. 

1 have already n\entioned that we measure the effcaivenc^ss of our 
foniud educatfonal system in the frameworks trf national and interna- 
tional surveys. Here, J concentrate scMy on iniemaik^naJ surtm^ — 
describing important methodok^ical and organizational conditions 
and some research findings. And finally, I discuss the significance of the 
international surveys from a national pt)int of view. It Is my view that» in 
the European culture, the international comparison of learning achieve- 
ments Ls not appreciated as much as it should be, given the impi)rtance 
and weight of the issue. Educational comparisons often deal with educa- 
tional philosc^prfiies, input faaors, and other issues of the cxlucational 
pHKess, but neglea the output side, Tlierefc^re, let me grasp this opptir^ 
tunity to speak about the importance c/ the comparison of output and 
its assumed advantage. 



Methods and Set-Up 

We have been a)nduaing international surveys under the auspices 
of tlie International /Xssociation for the Evaluation of Educational 
Achievement (lEA) for years. lEA studies involve students erf the same 
age, attending the same gnules, but studying m different educational 
systems. Their learning ^hievements are measured and compared to 
arrive at conclusions about development in the fields of education to be 
studied and the improvement of the whole schcxil system. By compar 
ing achievements and the learning environment, we can mutually lx?ne- 
fit from each others experience. This, of course, presupposes a high 
degree of J^dardazation of methods and the realization of the mutu- 
ally worked<>ut project. 

L b 
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Research findings are firsl publisted in an international volume as 
studies of the panidpating cxniniries. Interpretations ranj^ from simple 
descriptive analyses to multivariani maihematical-stati«ical analyses 
that umxiver cause-and-eflfea connections as well. The re!ittively sim- 
plest and most easily uncterstandable interpretation is the rank order of 
national averages. This sometimes sqjpears as if lEA were conducting 
some kind erf "intellectual Olympic Gam«." The international anal>'ses, 
however, show that ilye aim of the studies te to underst^id and improve 
the educational system trf the panidpating countries. Our account will 
rest primarily upon the rank ortter analysis 

The first lEA survey in 1970 tested re^ng comprehension skills 
and science knowledge. Mathematics was the next field of education to 
be studied in 1980 and 1981, and in 1983 science* achicvcmente were 
measured again. (This list is not complete as there were other lEA 
studies.) 



The Main Researdi Findings 

Reading Comprehension 

The 1970 findings of the reading comprehension study showed 
that I iungarian .students average achievements in all grades involved 
were generally below those of other lEA countries. It follows, therefore, 
that education in the Hungarian langua^^ and communicatton skills 
was one of the weakest fields in effectiveness. Subsequent national read- 
ing comprehension studies (in 1979, 1980, 1986) essentially supponed 
the lEA finding. 

The rank orders trf national averages, also indicating the range of 
the two inner standard de\'iaiions, in the international survey showed 
that at the 4th grade and 12th grade levels, Hungarian students were 
next to the bottom; at the 8tli grade level, they were eighth out of 
twelve. 

The poor results of the reading comprehension survey were for us 
like a bolt from tlie blue. Both pr<rfessional circles and the public 
reacted strongly. Experts mainly coasidered changes in le^hing meth- 
ods, and as a result, silent reading bcxamc ^entuaied. Also, we believe 
that the lEA findings contributed to the spread of diverse, alternative 
teaching methods of initial reading from the late '70s, Until then, only 
one teaching method of initial reading had been used, in ^ordance 
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with the cenual curriculum. These r^earch findings^ besides other fe- 
tors, also coniribuied to consider^dde innovation (rf education in Hun- 
garian laiiguage<xjmmunication skilk 

Madtemailcs 

The lEA has so far studied the efteaiveness d mathematics educa- 
iton twice:. Hungary did not panicipate in the first study in the 60s. In 
tl^ second survey, however^ the ^hievements of 8ih grade students 
(Population 2) were measured in 198C, and *n 1981 those erf 12th grade 
^Kknts (Bopu!atk>n 3) were nn^ured. Unfonunately, the international 
volume has not yet been puMished, and national evalu^ion stmiies are 
also missing. Nevertheless, we have rank order erf national averages 
from Amerkan sources at our disposal. They show that Hun^ian stu- 
ttents in the 8th grade are well above the international average and that 
at the 12th grade level, the top 5 percent score fifth out of twelve. 

Rank order shows that Hunf^ian students' adiie\'ements are usu- 
ally better than those erf the averages of other lEA countries, 1 hough we 
do not know the real causes and circumstances, the results clearly indi- 
cate the effectiveness erf" Hun^^an maths education in the period wlien 
we discaixied the traditional arithmeiic-geon^try tc;Khing for the sake 
of new maths ( 1975^ 1985 1 

Science 

The first international science siud>' tcx^k place in 1970 and the sec- 
ond in 1983. A preliminary international report on the laner was pub- 
Ushed in March 1988 (Sci^Ke Acbiei^^ment in SetHmteen Countries, 
ft^igamon Press) The main points of the bcx:^ were tnade public at 
press conferences in Lt>ndon and a few days later in Budapest. B<Hh lEA 
sdemre studies indicated, at least according to the arhievements scored 
in the test papers, that this field of education in Hungary can be 
regarded as effeaive. Our results stand the comparis^-jn with those of 
J^>an! This conclttsion is unquestionably backed by the folloviing rank 
orders: 

In 1970, the 4th graders were in the middle, the 8th graders sec- 
ond, and the 12th graders seventh. In 1983, their respective ranks nx^re 
fifth, first, and third (but with a larger number of Asian countries in the 
field) 

The relatively high achievements of the Hungarian students result 
from several floors. Among the most conspicuous are that the political 
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and prcrfessional conceras in science education coincided, that teadiii^ 
materials (curriculums, texdxxAs) are on a relatively high level, and the 
relative quantity erf time devoted to science teaching is high. Science 
education can, at the moment, be regarckd as tte prime branch erf Hun- 
garian education, thanks to the work erf schools, tejKhers, and audenis. I 
wouki n(X like to say, however, that we have nahing to improve. Fbr 
instance, we i«ed more student experimenLs and practical knowled^. 

Overall Coaclusions 

In our case, tte various lEA studies led to contradiaory results. On 
the oT^ hand, while it turned out that our science education and partly 
mathematics educaion are more effective than in many lEA countries 
that are more highly developed than Hungary, languagecommunication 
educatton, on the c^her hand, lagged considerably behind. So we got 
into a schizophrenic state of mind. Several publkanons came out to 
explain the reasons, and especially since March 1988, when the results 
erf the 1983 science study were puWished, there have fc«en fierce 
debates. The press, television, and radio all b^an to deal with the prob- 
lem. The interpretations called researchers' attention to new 
approaches and new aspects, and rer^archers couU justifiably argue for 
the importance of differentiated evaluation of the educational s>'stem. 
This was almt>sl unprecedented in the Hungarian mass media. 

The Benefit of Internatkmal Studies 

Based on the Hungarian experience, we can note three concrete 
advantages of empirical-comparative educational surveys. Fferhaps the 
most important outcome is that we have succeeded in gening accepted 
the principle that Hungarian students' level of knowlex^e can, and mast, 
be compared to intemattonal standards, that is, to the level of ck-vel- 
oped «)untries and not to provincial, political !>• set criteria. Many new 
facts and connections were revealed during the design and realizaiton 
phases of the lEA studies that contribute to the always-present fear of 
provinciali.sm, a natural consequence of the national character erf our 
education. Very often, it becomes obvious that a number of similarities 
exist between educational systems though they fimaion differently in 
various cultures. Thus the lEA-t>pe of international educational studies 
is an important means of improving the cooperation betvi^en the mem- 
ber countries. 
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Another Ix^nefit Ls alsi) obvious and conncacd with the previous. 
In light erf the results, ^t? can nu)re prirfoundly anal>^/.e the input-output 
factors trf educational systenifi. We can not only pornt out whether a par- 
ticular content or organizational faaor funaioas in a given culture, but 
we can ako make cx)mparLsons regarding ib- effectiveness. iEA studies 
have cxmtributed to the understanding of such issues as the effeitive- 
ness ctf comprehensive versus selective sch<x>l systems or the connec- 
tion between excellence and mass education. The more students attend 
school, the more talented children can be identified. Understandably 
though, the increasing numbers of attendance affecis the quality of edu- 
cation. 

A third merit Ls cona?rncxl with research methiKls. During the 
years of cooperation with IEA, we have adapted to our own circumstan- 
ces a a\seatx:h metluxJologv^ that nvakes ptxssible valid and reliable mc*a- 
suremmt oi learning achievements. In Hungar>', it has led to a new edu- 
catic^nal evaluation system thai can provide regular and reliable 
information to schcxils, politicians, teachers, and the public about tlie 
actual state of students knowledge. 



ERIC 



— 

50 



What lEA Studies Say 
about Cuniculum and 
School Ofganization 

EDWARD KIFER 

The international arena in general and tlw results of lEA studies 
in panicuiar are fertile grounds for ilK>se who wish to cite suc- 
cessful praaices that cteviate from existing ones. As suted in 
one of the fina official publications of lEA; 

If custom and law define what is educationally allowable within a 
nation, the cduc-ational systems be^wnd ones national l>t)undaries 
suggest what is educationally possible (Fcxshay et al. 1%2) 

French schtx^Ls are closed on Wednesday afterncxms. New Zealand chil- 
dren begin sch(X)l on the day of their sixth birthda\ not on the hrst day 
of scIkxjI that year Japanese mathc*matics teachers liave ajmparably 
lar|5e numbers ctf students U) texh, hut tiiey teach just three of tl^ six 
days of the school week. All children in Sweden are introduced to a str- 
ond foreign language. Elementary sch(X)l pupils in ilie United Stales 
recite a Pledge of Allegiance ^h morning. 

The list could be expanded almost indefinitely, hut these are 
enough o make the point that variatit)n in ix)licies, prtKXxiurcs, and 
praaices found in other educational systems provide a ct)unierfx)int to 
any one system's views and a hisis for change, innovation, and refc^rm. 
These differences are embedded in cultural contexts that act as 
restraints to their wholesale adoption. They serve, houevcr, as a beacon 
for what can be done. 
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Curriculum broadly construed ami diverse organi/aiion irfsclux)!- 
ing are mapr tliemes in lEA repons. The tairriculum noi only forms the 
backdrop lo lEAi cc)gniiive tests, but also is siTutini/ed ilK)rougWy ft>r 
its contents, resulting in innovative empirical meitsures of its implemen- 
tation. SclxK)l organization is a ftxiis, as w^^ll, since hisli)ries and scKial 
forces prtxluce ver>^ different tyi^es across tiie 40-odd systems studied 
by lEA in its short but pr^xluctive history 

1 deal first with issuer surrounding tlie curriculum and then with 
thvjse surrounding the organization of schixils. How schcK>ling is oi^- 
nized is intimately Ci)nnected to curriculum since differences in schcx^l 
structure are ^rften assiKsated with who in a sv^tem gets wliat kinds of 
experiences. The extent to w^hich a system s curriculum is differentiated 
;utd Ik)w that relates to sclux^l organization forms the third major sec- 
tion of this chiipter 

Issues Sumnmding the Curriculum 

The IKA af^proiich io curricular issues is scientific, outcomt^driven, 
and fiH used on finding commonalities among curriculums of the partic- 
ipating educational systems. It is scientific because the rationale for the 
studies involve s>^tematicaUy colltxting objective, empirical evidence 
about each system. Tlie ^q^proach is broadly IxtscxJ on views formulated 
by Ralph IVler in tfie 1930s and emphasizes behavioral spe< ifications of 
what is to be measurctl empirically 

It is outc^mie-driven because a major portion of the curriculum 
anah sis accompanying each surve\' is devoted to developing a content 
\'y beliavior grid to provide a basis for formulating an t)bjeaive lest 
(usually multiple cIkhcc) tl\ai can be used to compare cx)gnitive out- 
ctmies of the various systems. Whether it is a mathematics survey' where 
the content dimension might include, among others, arithmetic and 
algebra, and the objcxi:ive dimension wt)uld include, again among oth- 
ers, c^miputation and solving problems, or a literature sur^ry ih;n 
indudc^s sfu^cific texts as well as liieran' tlK'or\^ for content and (amiliar 
it>' with and expression of a consistent response to as the objectives, the 
grid forms a kind of Procrustean bed for the outconu*s t>f a survey 

For obvious reasons, lEA searches for comparability and ct>mnKM>- 
alities across tnJucational s\'siems, Obiu.iive, scientific study of educa- 
tional systems is based on the assumption that a)mmDnalitit*s exist 
among the systems and a sense of fair play suggests that comparisons 
should be made about thi)se things that are similar In the search for 
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common instances, diverse and unique phenonK-na must be left 
behind. 

As with all such work, hcwever, to the delight, one assumes, t>f edu- 
cational refonners, idiosyncratic praaia» and diverse opinioas spilled 
over the edges d" lEA sc- iice. Si> we not only have an opptirtunity to 
talk about things iliat seem more or less related to differences berswen 
systems (the scientific results), but also about evidence of unintended 
outcomtti of the studies. 

Time and the Opportunity to Learn 

Two related but different notions are central to lEA investigations. 
The first is the amount itf time allocated to different subject matter; the 
second, a unique and innovative lEA measure, Gpfxn-htnity to Utim, 
that aitempLs to define what students are ex{X)sed to in a particular i ur- 
riculum area. 

Amount oflInK 

A variabk', the amount of time alkxated to instruction, features in a 
number of lEA .studies, but is most central la results of the study t>f 
French as a Foreign Language. Carroll (197S) presents a series of antily- 
ses that, over educational sy.stc>ms, relate the number of >x*ars studying 
French to average f^rformance in reading, w riting, and listening. 

Figure 5.1, based on the data reponed by Carroll, depias a strong 
relationship between number of years of French study and French read- 
ing performance. That figure is similar to ilrose published in tlu.' lEA 
repon for each of the criterion variables and leads to an obvious conclu- 
sion: the longer .students study a .subjea, the more proficient they 
become in it. 

Carroll estimated that an average student would i;ike between six 
and seven >^'^^rs to attain a ".satisfactory" (and high) level of {X-rforni- 
ance in French. That amount tjf time could be redueeti if the student 
were fonunatc enough to {al have a fluent tcaclK-r (an Ci^.tt of aiiiu;M 
one )'ear); (b) l>e tauglit mainly in French, not in a mother tongue (an 
effea of a year and a Iwlf); and (c) have .serious aspirations to learn 
French (an efftxi of about a half war of iasiruaion) 

The implications for reform arc IkmIi obvious and .sensible. 11 a par- 
tkrular learning tiutcome is desired, then a frame or wimtow in the cur- 
riculum must provide a suffkient iimtiunt of time for the learning to 
occur That learning is enhanccxJ, and the size of the window tkvreased. 
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by the presence t>f effective teachers using appropriate methods with 
students who are highly motivated. 

The elegance of these findings should not be tost because they 
ai^>eal so wel to common sense. A curricular reform, for example, 
based on a fixed standard of performance, using empirical estimates of 
the amount of time needed to reach that level of performance, that 
could speciJy what qualities are needed by a teacher and wliat teaching 
meihofls are most effective, and responded specifically to the needs and 
aj^^irations erf" stuctents would, indeed, be a reform. Tliese findings, 
therefore, provide a beginning, a direction, and a challenge to reform. 

OiqKMlvnity to Learn 

lEA studies a miain an Oppt)rtunity to U-arn (OTL) measure that is 
an ingenious and innovative attempt to learn the extent to which a par- 
ticubr curriculum is implemented. Based on ;M:hievement te«s con- 
struaed faim the grids mentioned earlier, OTL is measured by asking 
each teacher in a survey to respond lo each test item in terms of 
whether or not his or her students have btx>n exposed to the material 
needed to be aWe to answer the question a)rrectly 

Figure ^.2 on page 56, from the science study (Comber and Keeves 
1973), shows the relationship between the OTL measure and average 
achievement across the various educational systems. While there are 
exceptional cases (the two bw ouilien* being systems in de\'cloping 
countries), there is also a strong positive relationship between achieve- 
ment levels and the opportunity to learn the material. 

This general finding also appeals to common sense. Students tend 
to learn what they are taught. Yet, all is not so simple. This finding 
occurs across systems that vary greatly in terms of national practicx-s, 
such as having or not having national curricuiums or common national 
examinaiioas. 

Reformers in a system wittwut a national curriculum wi)uld find lit 
tie evidence in lEA studies for the emcacy tjf a curriculum more cen- 
trally determined or more broadly applied. In the .second mathematics 
study (Bursiein in press), for insfantx«, both the Japanese and New Zea- 
land systems are based on natitinal curricTjIums. While the academic 
performance of the fonner Ls the highest of the 20 systems sfudied, the 
lattcr's achievement is mt^re modest. 

OTL, then, is a n>easure that speaks to the issue oi what students are 
exposed to in a classroom. Relationships between OTl. and achieve 
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Figure 5 - 

Oj^rtuntty to Loam and Average Achievement ^ross Systems: 14-YearOids In Science 
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mem are relatcxi to the amount trf tx>nt« ni an-ered in dassrtxims, ix)t 
necx-ssarily to the extern the curriculum is standardized. Am«hcr way to 
state the finding is to distinguish between curricular aims at whatever 
level they are specified (local, province, or s^Titem) and what a teacher 
actually does in a classroom. OU reflects what Is implemented; r;ot nec- 
essarily what is desired or intenited. 

Results of Oil, as well as other related analyses acro.vs sun'e>'j: and 
systems, suggest also that the anwunt of material covered, na tlx- 
sequence, is a predictor of levels of achievement. Several sets of results 
point to the secondary importance of seijuencing. In the "feasibilit>' 
study " (Foshay et al. 1962), correlations between item difficulties across 
systems were produced for four subjea areas-, maihematics, reading, 
geography, and science. Figure 5 3 presents both a stem and leaf display 
and boxpltH erf the correlatioas between item dilficullies for the geogra- 
phy test. The airrclations between these item difficulties should be uni 
formly high if different systems approach the content in similar ways 
0.e., that sequencing of material is as rnfxmant aj. amt^unt of material 
awered). While iherc are differences l^etwcen subject matter (mathe 
matics correlatioas tend to be higher than the others), the results are 
notable for variati<^n Ix'tween systems rather than similarity 

Results of other surveys, too. suggest similar things Both the first 
mathematics .stud>- (Hus^^n 1%7) and the scientv stud>- repon single 
item results horn systems in whicfi a panicailar item js cjiIkt morx.- difti 
cull or easier than expwxicd if performance a:rass the item set were 
consistent between counirief,. While the literature survey (Purves 1973) 
found that oppt)riunity u; learn is important in terms of explaining 
achievement difierences beti^'een systems, students in iho.se SNsiems 
typically respond differently to the various reading pass;igi-.s. Thai is, 
what they are taught is related to achiev ement differences, but Ik w itie> 
are taught is not. 

Evidence for the negative effea of redundancy, a counierjx)int to 
OTL {too much time spent on uk) tiarrow a .set of outcomes and nui 
enough expixsure to new topics), conies from twt) recent sources 
McKnight {198"^) pegs the relatively low performantx- of U.S. students 
on mathematics achievement tests in the setx)nd mathemaiics stud> to a 
curriculum that moves very slowly cxHitains huge amounts of redun- 
dancy, and, compared with other systems in the study, introduces fewer 
new topics. Kifer and Wolfe ( 1985) found a correlation lx.'tween preie.si 
scores and OTL This demonstrates tliat students exposed lo material in 
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Rgure 5.3 

Stem and Leaf and Boxplot of ConBlattons Between Kern 
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a previous >var hav e a higher likclihvxxi of being exposed lo the male- 
rial again ihan do students who have never txx*n exposed to it. 

Again, ifK OTL results suggest thai the amount of material covered 
counts more tlian how it is covered. Thai, of ci>urM?, makes a gcxxl deal 
of sease. While students do not learn all they are i iughi, they do not 
learn wlia( thej have nc^ Ixren exposed to. 

Summary 

T\vo crucial alterable variables of the IHA surveys are the amount of 
time allcK^aied to a subjtx^ and the op{5oriunity to learn important fea 
turcs of it. Although bc^th of these measures are related to how educa- 
tional s>^tems use time, they have different implications. 

Where CarrolFs Findings include implications for tlK* aUcxration ot 
cinK?. tether a)mpetef ice, and te-iching methcxlology, the Oil findings 
suggest that the relative importance of the variables place expi>sure to 
content very high on a list and teaching variables and "sequencing" of 
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the curriculum rather low. This suggests that ihase interested in curricu- 
lum reform should wuric diligently to include average erf "desirable" 
c<jnient in educational sj'stems, but worry- less about how the content is 
handled once there. Exp«»ure to the material is the necessary randition 
for learning. Sequence, pacing, and teaching variables follow in impor- 
tance. The mere existence of the subjea matter placed before students 
is a crucial ingredioit trf any tnirriculum reform. 

These findings enwrge from a "scientific" view how to study 
diverse systems Internationally Such findings mute diffe>ences 
between tte contexts in which scluwLs t^rate and emphasize common 
aaivities that can be put into operation and measured more or tess pre- 
cisely 

Educational aaivities in schools are not necessarily conduaed in 
optimal ways. For example, the average French child acxjuires compe- 
tence in his or her language in two years, but it takes seven in other 
educational .systems to produce arguably less fecility For the French 
child who masters the language in two years, the issues are not those of 
learning French nor how he or she is taught. tk>minating iht»e two 
years is a .social world that the child is trying to understand. 

Not only are educational systems different from a sKial world in 

■ ms of efficiency, but they also vary in the extent to which scientific 

■ Jels apply to what they do. In each of the findings cited, some sys- 
cms deviate substantially from what is found on tlx* average. Typically, 
s>'stems in developing countries do not perform as predicted by a scien- 
tific model. Other systems, for example Rumania in the French study, do 
much better than a mcxiel would predia. This reminds as that any 
reform should be as cogni?ant of the needs of stui^nLs, the idiosyncratic 
asfXHTs of a social content, and tl\e peculiarities erf a culture as it is of the 
scientific findir^g?!;. 

These fii»din>«j, then, reprf'sent a stalling point for curricular 
change and reform. They do not define the limits of wliat tan be 
changed or how such change sljould be eficaed. They point to a way 
for a reformer to begin to think about such matters. 

School Structure 

F:ducational systems surveyxid in lEA studies vary dramatically in 
their .structure— htw they are organizal and who they serve. Schtxil 
siruaure and its implications formed a major focus of early studies. 



CuWUCllUtM AND SCHCXX. ORGANiyjOKSO 



Interest waned during the period o( tlie six-subject surveys, but rekin- 
dled in the latest survey's. While the next section deals with the inter re- 
latedness of school struaure and curriculum, the fcKus Ls primarily on 
Issues raised by different types of schools in the various s>'Stems. The 
following three seaions tfcal with findings related to type <rf scIkxjI, 
selectivity and retentivity, and ihe ntxion of yield trfa system. 

Type of School 

Virtually all the lEA surveys measured the type of schtx)! in which 
students were ennilled. For ;K:ademic purpcxses, such typc*s could \'ary 
from aimprehensive schtxjls (all students attend the same type of 
school) to selective schools (schools where only an elite can enroll). In 
systems with seleaive sc1kx)Is, other types of scIkx)1s are usually availa- 
ble to students not among the academic elite. Examples include schools 
with a vocational or technical curriculum. 

Systems vary m)t only by types of scht«ls, but also in terms t)f when 
students are channeled into the different schotjls. For example, by age 
13 students in Flemish Belgium are enrolled in different types of 
schools. In the United States (at least at the descriptive level) all students 
air enrolled in comprehensive sc1kx)Is for as long as they attend sch(x>l, 

Tlie variable sch<x)l type figures prominently in IF-A publications, 
especially in the six-subjea sur\'i?y, wliere it was included among a 
bkx:k ol variables in a series of regression analyses. It was a powerful 
prediaor ()f ^hievement, mainly becaase systems, on the average, 
place more or less capable students in different types of sch(X)ls While 
school t>T5e is a major and pi>werful prediaor of achievement, its impli- 
cations can be traced through two different and related issues popular 
in l¥A vt>lumes 

Selectivity 

One major issue addressed by the first mathematics study was to 
discover the circumstances under which the most talented students 
develop best. The argument for selective schtwls is that talented stu- 
dents, learning together, will not be held back by less talented members 
of tlK'ir ctjhort; tlwrefore, to develop the bt^t talent a system should be 
selective not comprehensive, 

The analyses of the data, however produced a rest)unding condem- 
nation of seleaive sehtx)ls. Although average scores in seleaive educa- 
tional systems were higher than averages in txjmprelK-nsive schtxjls, a 
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vanety of analyses show that the "best" students did equally well regard- 
less of the type of school in which they were enrolled. 

While there is no obvious advantage of elite scIkx)1s in terms of the 
highest performing siuctenis, a con^rehensive system benefits ger^r- 
ally if the criterion is the yield of the system (Husdn 1967). That is, if one 
could posit an iralicator reprej^nting the sum of mathematical knowl- 
et^ (as opposed tc the average amountX then comprehensive systems 
are clearly preferred. Afore students knowing some mathematics pro- 
duce a higher yield than a few students knowing a kx of maitematics. 

The finding from the first science study are equally unambiguous 
on questions related to sdeaiviiy of educational systems. A strong 
mtxtel to predia average achievement levels includes a large adjust- 
ment for selectivity, but small adjustments for differences between sys- 
tems for the performance of top audents. These results suppon those 
mentioned earlier, comprehensive education appeais not to harm the 
highest ^Thieving students and increases the overall amount of kncjwl- 
edge produced by an educational system. 

Retentlvlty 

ReientiviLy, another Lssue addressed in lEA studies, is related to 
seleaivity and yield, but focuses on a different set of questions. Whife 
most developed countries have compulsory schooling ur.il at least age 
16, a varied percentage of the student cohon attends some t>pe of 
school until time to enter a college or university At the terminal year of 
secondary school, anywhere from about 10 percent to 90 percent of the 
cohort Is enrolled When who is still in .school varies so dramatically 
betwev-n systems, it rames as no surprise that those with a small per- 
centage of students still in school have higher achievement averages 
than those that have high reteniivity— a larger proponion of the cohort 
still enrolied. Rather the question is whether enrolling more students 
in school lowers the achievements of a few. Again, the lEA results across 
surveys suggest "No." While increased retention t)f .students doc^ influ- 
ence negatively average achi -'ement values, it appears to have no sub- 
stantial influence on the hig st .scoring portion of the achievement dis- 
tribution. Hence, a .system can increase the number of students who 
complete secondary school without harming the education of an elite. 
The net result of Increased ref ntion is lower average .scores, more 
knowledge produced, and no '4)preciable influence on the highest scor- 
ing group. 
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Summary 

lEA results suggest ihai an expansion c>f alHX)ling should be both 
ccNiipreheasive and nonseleaive. The system would prcxluce nK>re 
knowledge witlwut penalizing iLs mosi cap^Ie siuctonis 

The findings are based on assumpttons, Ixnvever, that more sckx>J- 
ing for longer periods of time for more students repn^nis defcnsiWe 
needs erf the young and Ls a desirable goal for an educational system. 
Oiailengers to these assumptions ai^ue that more schooling for more 
people is both unnecesrary and fundamentally coercive: many otlier 
settings are available in which people can learn; the more technological 
the society the less schoohng needed; and, adolescents nt^ real vmxk 
and real experiences rather than more schooling. Implementing 
change based on lEA results on retention and seleaivity, then, should 
consider first the assumptions u|X>n which the aiguments against selec- 
tivity and for mcreased retentivit>^ are made. 

Curricuhun and Schcnd Structure 

Psotic^^ation ^raus Achievement 

Issues surrounding seltxtivity and retentivity push the educational 
reformer away from questions about achievement and talent as criteria 
for accomplishment to a position that makes panicipation in schooling 
and the terms of that participation major focal points. If one wants to 
educate only an elite then tk? system should be seleaive and retain 
fewer students. Only after the decision has been mack that the masses 
are capable of being cxiucated and making a commitment to educate 
each individual does selectivity- look undesirable and retentivity become 
a goal. HencT, participation in schcxiling becomes a criterion for a desir- 
able educational poUc>: 

Participation, however, is an ambiguous term when it comes to 
schtx>ling. A system can cntsily cfemand that students be present in 
schools without c^Tering them the best the system has to offer Systems 
which are neither elite nor specialized and have high retention rates 
can still rcserw the best schcx>ling for a small number of students 

llrackii:^ and Sorting ^udents 

Tracking and sorting of students becomes an issue similar to selec- 
tivity when retention is high and schtx>ling is k)ng because mH all stu- 
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dents are offered opptjrtunities to panicipaie fiiUy in schmJ life. In his 
feasibility study, Pidgeon (1962) used the standard deviation of test 
scores to draw condusioas about the effects of "streaming" in England. 
While trying to understand the low average performance but great vari- 
ance in achievement, he concluded: 

PSerfiaps exerting a greater influence, bRiwevei, is the belief a 
teacher may have that innate ability is erf paranKJunt imptjrtance in 
determining ihe level erf attainment to be expected ko.ti a child. 
Streaming by ability, which is viewed as an administrative device 
resulting from the acceptance of this belief, will merely tend to 
enhance its effects. When all these faaors act in the san^ direaion 
the effect will clearly be greatest and this is what happens in 
England, Here, it is claimed, the aims and, more especially, the 
beliefs of most teachers and educational administrators lead them 
to expea wide differences in f^rformance, and this is what is 
therefore achieved. Where, on the other kind, tlte grade place 
ment system operates and especially where, within such a system, 
teachers do not attempt to measure innate ability and therefore do 
not expea their pupils' attainments to be matched to it, then the 
dispersion of achievement will be much less. 

Whatever educational assumptions underlie the use (rf tracking and 
streaming, tliese practices have clear OTnsequences in fcrms erf curricu- 
lar issues. The flip side trf tracking is a differentiattxl curriculum. 

Differentiated Cuirkrulum in Grade 8 Mathematio) 

Anal)'ses from the Second International Mi ihematics Study (SIMS) 
produced a substantial number of resulLs addressing the effects <rfa dif- 
ferentiated curriculum in Grade 8 (Bumein in pressX A pretest was 
administered in eight educational systems at the b^inning of the 
sdtodl year, making it possible to describe Ikw students are allocated to 
classrooms or schools within those systems. In aniMher paper (Kifer in 
press), I describe how the variance decompwition ctf Figure 5.4 refleas 
decisions ma<fe to track students either into homogeneous classr;x)ms 
or different types of schools. 

In Figure 5.4, areas of the circles are roughly proponional to the 
total vari'ince in achievement for each system, The wedges within the 
circles represent percentages of tt)ial variation found between students. 
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classrooms, and schcxila The circles coniaining only two wxxlg^ depict 
systems thai did not sample two classrcK>ms per school in those cases, 
the variation is labeled student and sdxxjl mruUton akb.)ugh, tlwreti- 
cally, the wedge for school rontains both the classroom and schcx)l vari- 
ance. That is, the l>etween-classroom variation, if any exists, is pan of 
the between-school variation, not the between-student. 

Iht differences between systems are dramatic. Not only are the 
total variances (indhridual differences within a system) erf strikingly dif- 
ferent magnitudes, but also that variation (how tlw system responds to 
individual differences) is divided In distinct ways. In Japan, for instance, 
almost all of the large tcrtal variation is between students. Since such a 
small amount ctf between-schwl variation exists, variation tetween 
classes in the same sctool must likewise small 

/^>parently, the Japanese either ignore indhridual differences when 
assigning students to classrwms or their educational {wlici^ produce 
equality among classrooms and schools. Neither homogeneous group- 
ing in mathematics nor sorting by schot>l takes place in Japan in Grade 
8. 

The US. provides the most extreme contrast to the Japanese pat- 
tern of variation. Tl^ m^nitude of the between<lassroom component, 
which represents tracking in US. schools, Ls its single largest comjx)- 
nent and exceeds comparable values in all of the other systems. 

While the distinaive patterns of other systems have been discussed 
elsewhere, the m^or conclusion is worth repeating. These eight sys- 
tems demonstrate a w^ide variety <rf pr^ica; in stjning students into dif- 
ferent classrwms or schtx^ls. They range from what is done in Japan, 
France, and Ontario (where no sorting has yet occurred by Grade 8) to 
what is done in the United States where more variation in achievement 
Is aJlcxated to wrting praaices than to individual differences, 

Coriseqtiences in the US Since the VS. shows the largest effea due 
to tracking, it is wonh reponing what is known about the ronsequences 
of ihepolicies that produce it A typical pattern is to form, In any school 
with a sufficiendy large enrollment^ four tyjx^s of mathematics class 
rooms — algebra, prt^algebra, general, and remedial classes, The typical 
rationale for this Isthat homogeneous grouping provides better instruc- 
tion. What is igiK>rc^ isthat the practice also produces fundamentally 
different patterns ofopportunity, panicipation, and exposure. In Grade 
8, some students areexposed to mathematics content and experiences 
that others may never get. This is opposed to what happeas in other sys- 
tems with no comparable tracking. 
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Figure 5.4 

_ Variance decai^itiwi ct the populatit^ A (grade 8) pr^st core lest for each of the eight toigitudmal systems The area of the 
p« IS rajghly proportiwial to tfw size of the overall variarce in each of the systems The 'wedges' of the pie reflect the proponions of 
tt^nance that are attributable to difterencos between students within classrooms, classrooms within schools, and between schools 
TTwse systems that have only twro wedges did not sample tvw) classrooms f»r school: hence, one cannot distinguish between the 
Classroom and school variances. Given the nature of the samptes, however, tte profX)rtion d the pie attributable to school differences 
contains tfie dassFoom differences. 

Japanls pie is larger than tJie others in part tecause a kwger pretest. Theirs had m items versus the 40 items for the other -.vs- 
tems J3ther analyses suggest that with a test of 40 items, the Japanese variance would be about equal to or perhaps sliqhtly laruer 
than that of the USA. w j j 
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Figure 5 5 contains a set erf boxplois shewing OTL ratings by class- 
room type, it LS evident from Figure 5.5 that tra.-king students and differ- 
entiating the curriculum are two sides of the same a)in. Students in 
algebra classes are exposed to very different kinds erf material from 
those in other classes. Students in remedial classes are exposed to very 
little mathematics in comparison with txher groups and are likely to 
have been tracked so that they never will be exposed to the material 
provided to the algebra students 

These practices deny c^ponunities ft>r tlie many to experientv 
what the few get, plus other equally undesirahk" side effetxs (Kifer 
1984). These include the following: 

1. Sorting is replete with misclassifications. Only about 50 percent erf 
the top 10 percent of students are placed in algebra, the enhers are 
^read among the remaining class tyjK's. 

2. Seming favors whitt women students from wealthv honx"s If placv- 
mem in high level classes were strictly meritocratic, more boys, 
members of ethnic groups enher than white, at. ' more children from 
ptx)r familiers would be in those clas,st?s. 

3. Selection is se) rigid and .so early that it etfettivcly eliminates the 
major proptmion of the a)he)n from participating in the most 
advanced mathematics t)ffered by the schex)! system. (This is dis- 
cusset' in me)re detail below.) 

All s( h<H)l systems, not just the U.S., son students. These results 
shew the effetts in a system tliat dtx^ the earliest and greatest amount i >f 
tracking. Others tend te) son later in a student s carei-r. No system stud- 
ied by lEA docs no seining. 
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Figure 56 shews that issues surre)uneling participation in Kipula- 
tion B (the terminal ^tfar of .*icvondary schix)l in SIMS) are ver> different 
from those of Populatiew A (tirade 8). Virtually all students take sonx- 
type erf mathematics in the 13-year-erfd population. By the end erf sec- 
ondary school, defjending em the system, a large propeirtion erf ilie 
cohon is eithei ne) longer cnarfled in school or not taking mathematics 
or both- 
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Rgure 5.5 

OTL by Content Arra for US. Class lypes 
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Partlcifmtion in Schooling and Advance Mathmnatics Population B 
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The figures at>ove represent the percentages of tt^ cohort that are stiti in school sayd that which is taking advanced mathematics 
courses Tf® average age. rKimt>er of years of schooling, and definrlk^i of advanced mathematics vary anwig educational systems 
Students in Ontario Q'Z atxxil one year otder bt^ are enrolted in grade 13. for instar^e Although in rrost systems calculus is the stand- 
ard content, that differs as well In the United States, for example, (^ly about 2 percent of ihe cohort is enrcHled in a calculus course, 
less than one-half of them are enroled tn advanced placement calculus. 



The estimattxl pcrceni of ihc a)liort sfW in st*hcx)l ranges horn a 
high ViJue trf over 90 in Japan to lows of n in New Zealand and 
England Hk- percx>ni taking advanced niathcmatics a)urses ranges 
from 3 high of 50 percent in Hungary to kws trf6 peaenl in Israel and 
New Zealand. 

77je Hungarian Example While most systems are very selective at 
this level, Hungary is a striking exceptitw. Although "only" *>() }X'rcx*nt 
of the cohon are still in schcxil, all of them take advanced mailK'niatit-s. 
This system apparently is telling students itiat ^anaxl mathematical 
knowledge is so important that every student mu« have it. 

Miller and Unn (1985) examined achievement iMiterns in light of 
different retention rates and reptin f t) things that are relevant. First, 
although the average level <rf achie .?ment for Hungary's students is 
dase to ihr - bottom among the sysiei it is very ck>se to the ILS mean. 
Second, the top 1 percent and top 5 jwrceni of Hungarian students per- 
form near the top of the distribution of scores for the systems in the 
stud)' From a panicipation perspective, the Hungariaas have it Ixwh 
ways NtM only do iliey provide advanced mathematical experiencx-s to a 
large pea-entage of the Ckhon, and thereby increase dramatically the 
sum <rf mitthematical knowledge in the culture, but they also dt> it with- 
out sacrificing the talents of their most capable students. As a mtxlel for 
bc«h providing opponunity and creating a ptx)l of talent, Hungary's 
bears scrutiny 

The U.S. Case. Since the U S. case is praaically the opposite of the 
Hungarian one, it again cteserves scrutiny The United States has a high 
retention rate and an average jx'rcentage of students taking ad\'anced 
mathematics. The latter count, however, is misleading. The curriculum 
is differentiated at this k'vel as well. Figure 5 7 shows Ixnv tlK" U.S. 
stands when content areas are broken broadly into cakulus and other 
courses. The results suggest thai a smaller proportion of the cohon 
takes advanced mathematics (advanced mathematics is the cakulus in 
virtually all systems) than in any other s>T;tem. Since calculus is standard 
fare for the other systems, the U.S. percentage is really much lower than 
it appears. Calculus courses can be bmken down even further to tho.se 
considered advanced placement courses and tht>.se that are no; If the 
advanced placement calculus were considered life in the fast lane, then 
an extraordinarily small numl>er participate in the lx*st of .scluxiling in 
the U S. 
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Figures.? 

VS. 12th Grade Partlclpstlon in Schoolii^ and IMathematics 

Perceniages of the cq*w1 in the United States enroited m school, advarwed mathemaitcs classes, and calculus, respectivt'iy The 
calculus cxxirses can be divided turfher into calculus and advanced placement calculus Courses othrr than calculus arc composed 
of mixed content matter Population B rs defined as those students in ttie final year of secondary schooling who are enrolled in malhe 
matics as an integral part ol university preparation 
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£arly IVacking 

Hew early and to what dc-pav systems sort and track students is an 
issue trfn«aji:?r a7nsix|ueiKe:» for vvh.ii a system can pnKiiice, If Pndings 
about mathematics repiesent what tiapjx-ns in scIkhiIs, early tracking of 
students has a profound efftxi on chances for many to be expost*d to 
learning experientx's offered to a tracked elite. Hy GrdLiv 8 in the U S , 
for instance, Ie*;s than IS jKivent of the students are in a track iliat will 
alk>w tlK*m, ft>r iastance, to take cak ukis in Ciraik? i2, mj there is no way 
that sv-stem can produce as much knowkxlse as do systems without 
early tracking. The practice of tracking so early etft\tively eliminates 
the f possibility for most stULfonis to ex|XTience what is considered the 
best a sch(H)f system has to ofler 

Early tracking, and its conse<iuences, appears from an IMA }xrsjxv- 
tive to be an issue ver>' similar to tliat i>t an earlier oix» irf select iv it wer 
sus comprehensiveness. Elitisn> is less evident in a comprehensive orga 
nizational sclx^me with high retentivity But it is no less questionable in 
a sixial arena where issues of ;K'hie\ enx-nt, opfx>n unity, and pariiiipa- 
tu)n are im}X)rtant. 

Suih issues, given the earlier caveats alxmt l¥A sci.-nce atul tlx' 
assumptions u|x>n which it is bascxl, are most salient when one has the 
benefit t>f an international {x.^rs}xrtive. Tht*v can Ix* addressed with the 
evidence providcxl by lEA surveys. l\.*ople ititeresttxl in change, reform, 
or innovation should continue iheir W4)rk witli the hcnefii ot evidence 
from the lEA exjXTience 
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What BEA Studies Say 
about Teachers and 
Teaching 

IjORINW ANDERSON 
T. NEVILLE POSn^THWATTE 

From the earliest lEA studies to the present ones, researchers 
have been interested not only in describing student achieve- 
ment and the differences in achievement both within and acrcxss 
countries, but also in understanding the reasoas behind these differ- 
ences. Their search has led them to consider the influence of teachers 
and teaching pr^rtices on student achievement. 

^n the following discussion, we will draw heavily from three genci - 
ations of lEA studies: the first mathematics st. dy (Husdn 1967), the six 
subjea surveys (Peaker 1975, Walker 1976), and the Classnx)m Environ- 
ment Study (Anderson 1987; Anderson, Ryan, and Shapiro 1989). 
Aldiough the initial studies were surveys, the Classroom Environment 
Study relied exieasively on classroom observations to aaswer two 
major quesiioas. First, how similar are classroom conditions and teach- 
ing prartices in countries throughout the world? Second, which, if any, 
of these conditions and praaices are associated with higher student 
achievement and more positive student attitudes? Funhermore, while 
United States students were involved in the first i^ries of studies, no 
United Slates classrooms were observed during the Classrcx)m Environ- 
ment Study As a consequence, American readers may be able to "dis- 
tance" themselves from the results without feeling a need to explain or 
justify them. 



Unin W. Atiderson is Researh Prcfessor vf Education, Uniivrsity 
South Carolina, Columbia. Netille Postlethtmite is Professor ofatm- 
pamtive Education, l/nivemty of Hamburg UPfes/ Germany 
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Whai his emergeti from rhe^ studies is a r»ei erf generaiizatioas 
about teachers aiid teaching that hold prc^nisc for undc^rstaiiding and 
improvement. These generalizations apply to students, teachers, and 
dassrcx)ms ' on the average," rather than to any panicular student, 
teacher, or classnx)m. Exceptions quite definitely exist and may, in faa, 
abound. To place our generalizations within a proner context, we will 
first describe what lEA studies say alx^ut the relative tx>ntributions of 
homo and schix>l to student achievement. 



Home, Schcxil, and Student Achievement 

In combination, the results of the lEA studit^ supix^rt the well- 
krnwn fact that students backgrounds exert a jxwerful influence on 
their achievement. The studies, however, go bevxjnd mere replication of 
this and provide several explanations for it- 
First, far greater differences exist in the homes in which students 
are reared tlian in tin? schools they attend. As Ft^silethwaite (1987) 
asserts; 

After all, teaching is not like parenthood- There are no entry qual- 
ifications to become a parent, while in many a)untries iliere are 
strict entry qualifications to join the teaching profession. Conse 
quently. althc^ugh teacher diflereixres often look laige to f^udents 
or the r parents, these differcnces are, in fact, small compared 
with differenofs between homes, Thas, with little variability 
between teachers, it will be more dilficult to detect teacher effects 
compared with (home l>3ckgroundJ efft*cts (p, 1 ), 

Second, the impaa of the home on achievement is often amplified 
because studenLs from different backgrounds are sent to different types 
of sc1kk>Ls (e g., private versus public comprehensive). In several of the 
countries included in the Classixx:)m Emironment Study, the type of 
school that students attended was strongly relaic-d to their achievement. 
In thc^ countties in which this finding was not supported, only one 
tyf^ of school was typically included. 

Third* the impact of home on achievement is also amplified by dif- 
ferences in the aspirations students bring to school Students from 
middle to upperdass homes tend to perceive education to be of great 
value and seek nwre of it. As a consequeno \ tln.'y tend to enter j^*hools 
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aiid classnx>nis Ixntcr prepared academically and expend greater eifon 
in learning. ThejMf initial student differences are strongly asscx-iated 
with differences in overall achievement (Anderson ei al. 1989). 

F nally, the impact of home on achievement is different for differ- 
ent suhjects. The relationship hetween background atKl achievement 
appears to be strongest for subjects such as reading aimprehension, lit- 
erature, science, and the ct^nitive aspects of civic education. This rela- 
tioaship is much weaker for foreign languages and the affective aspcas 
ctf civic education (Walker 1976). 

While the lEA studies shed st)me light on the reasons for tlie 
strength of the relationship between home and achievement, they yield 
no evidence that would deny that relationship. Home background, 
either directly, or indirealy through its effea on aspiratio,.s and initial 
arhievement, influences student achievement in mtist suhjeas. Furtlier- 
more. because t>f the strength ttf this relationship, the impact of school- 
ing variabk?s (that is, those pertaining to teachers and teaching) is likely 
to be quite small in .studies, such as lEAs, in which students from a wide 
variety of backgrounds are included. As a consequence, when interpret- 
ing these results, one must remember that associations between teach- 
ers, teaching, and student achievement that are apparently small in mag- 
nitude may be quite significant. With this caveat in mind, iet us miw turn 
to our major generalizations. 

Genemlizatkm l—Greater teaclxr experience atid stthject tmiter 
knowled^ are associated with greater skills in classroom nutnagement 
and biglwr lerels of student adneivment. 

One of tlie few findings consistent acn>ss all countries in the Class- 
room Environment Study was that experienced teachers spent less time 
on classroom management, less time in classnx)m management, in 
turn, resulted in greater achievement by their students (Anderson 
1987). 

The relationship between tea:her knowledge and student achie%'e- 
ment was supported in several of the studies in the six-.subjea survey 
(Walker 1976). In the science study for example, science teachers 
respoasible fo*- teaching 10- and 14-year-old students in one ctnintry 
had lower achievi?ment than the 10- and 14-year-t)ld science jtf«<*vi» in 
another cotmir>'. As .night be expected, the avei^ge ftucfcnt achieve- 
ment in ihe fin>t country was much lower than that in the second. Simi- 
larly, in the stud" if French as a foreign language, student performance 
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in reading, listening, writing, ami speaking was strongly related to 
teachere' ratings erf their own skills in each of these areas. 

The finding concerning teaching experience is consistent with the 
developmental model of teachers proposed by Fuller and Bnwn 
(1975X In this model, less experienced teachers are concerned primar- 
ily with personal survival. As teachers become more expericnaxi. they 
become more concerned with the acquisition and refinement of te.K:h- 
ing techniques and, ultimately, with the development and learning of 
individual students. Therefore, novice teachers are likely to spend more 
of their time on classroom management. Without proper management, 
their very survival is threatened. 

The finding concerning teacher knowledge is related to what 
Shulman (1986) refers to as the "missing paradigm prob em. ' i*'Xord- 
ing to Shulman, the recent trend in research on teaching has been 
toward the identification of generic teaching behaviors and skills that 
transcend the boundaries of subjea matte- knowledge. Knowledge of a 
particular sut^ect and the ways in which this knowledge can be trans- 
formed by the teacher to make it "educationally digestible" to students 
are ms^or components of the missing paradigm. The relationship 
between teaclier knowledge and student achievement is supponed by 
several of the lEA studies. The relationship between teachers ability to 
transform their knowledge into forms that students arc most likely to 
understand and stuctent achievement remains to be explored. 

This generalization crfJers at feast rwo implications for teachers and 
teaching. First, teachers with different levels of experience are likely to 
have different needs for inservice training in topics such as classrcx)m 
management, instructional strategies, and the like. This poinis to the 
need to individuali:^! inservice training. Second, increased attention 
should be paid to the subjea matter kmwledge that teachers bring to 
the classroom. This attention should focu.-> on both iIk? subjea matter to 
be transmitted and the ways in wni.h it can be transmitted in order to 
ensure high levels of achievement by large numbers of students. 

Generalization 2— -Students who ixuv greater (^portimities to leant 
the knoieiedge and skills incltuied on the adnetxtnent tests are likely to 
bat>e biglxr ieivls of adneimnent. 

This rather commonsense sratement was supponed in the majority 
of countries in all three sets of IFj\ studies. In fact, oppt^nunity to learn 
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was one of the first scMing variaWes included in the lEA studies 
(Hus^i 1967). Opptmunity to learn" was defined as teachers' ratings trf 
the extent to wliich tlieir students had received instruction in the 
knowledge or skill associated with e»:h item on the achievement test. 
More specifically, the opponuniiy io-k^n index for each class erf stu- 
dents was the percentage of items that tested knowledge or skills that 
the teacher believed he or she had taught. The higher the percentage, 
the greater the opponunity to learn. 

In the Classroom Environment Study, we made a second estimate 
of opportunity to kam. Mandeville (1984) suggested that students in a 
classroom could be said to have had an opportunity to learn the knowl- 
edge and skills tested by an item if five percent more of the students 
answered the item correctly on the p^wjt-test than did on the pretest. 
This index of opportunit>' to learn (referred to as the fiveperceni rule) 
was the percentage of items for which the increase in tte percentage of 
correa aaswers from pretest to post-test exceeded five percent. As 
might be expected, the estimate of opportunity to lewn provided by 
teachers (termed the "judgmental estimate") typically was higher than 
the estimate based on the five-percent rule (termed the "empirical esti- 
mate"). The true opportunity to learn is likely to be somewhere 
between these two Climates. 

The concept of opponunity to leani fias . least two implicatioas 
for teachers and teaching. First, oppt)rtunity to karn is a potentially 
powerful extraneous variable. That is, differences in the effectiveness of 
teachers and teaching practices are likely to he hidden by differences in 
opportunity to learn. Students having minimal opportunity to learn 
what is included on a test are likely to perform poorly on that test 
re^rdless trf either the quality of the teachers or the quality of their 
teaching. 

Second, and as «..)nsequence of the first implication, educators 
would be wise to attend to issues surrounding opportunity to learn as 
they struggle to understand and ultimately improve the quality of teach- 
ers and teaching. In panicular, the concept of "aligiiment ' is relevant. 
Stated simply, alignment is the extent to which the gtials to be achieved, 
tests to be administered, and instruction to be provided are consistent 
with one another. In order to maximize alignment, one typically begins 
by identifying and holding constant either the goals, tests, or instruc- 
tion. Thus, one can have "goal-driven" alignment, "test-driven" align- 
meni, or "instrurtion-driven" alignment. As educators, we all have our 
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personal preferences concerning the proper staning point of these 
alignment efforts. The lyitr rationale (1950), for example, emphasizes 
the need to begin with dear goals. Many experienced teachers, how- 
ever, tend to b^n either with textbodts or instruaional atti\iiies 
(Clark and Peterson 1983) Regardless t>f the staning point, however, 
alignment needs to be addressed. Increases in alignment will quite 
likely re»ult in ir-Ttiases in r^ponunity to learn which, in turn, are 
likely to affect student achievement. 

Getwmlization 3—Vx grmter the amount vf time spent i tt teadjing 
and learning, the greater the achieivment cf studenix. 

At first blush, this generalization seems as ■ common iral" as the 
previous one. However, it is far more complex. For exa.. jk, the lEA 
studfes produced no evidence that increasing the nnmlHr . f hours per 
year spent on the study of a subject produces an increase in achieve- 
ment. On the other hand, the number <^ years students .spend studying 
a subject such as French has been as.sociated with proficiency in that 
subject (Carroll 1975)- 

The amount of time spent on learning i.iay be extended by home- 
worit. In faa, homeworis was one of the few '.variables in the six-subjea 
survey that was coasisiently and positively related to stuttent achieve- 
ment, even after differences in home background, type of scnool, and 
other variables were taken into coaskieration (Peaker 1975). Pacing, 
defined as the amount of content awered per unit of time (e.g., per day, 
per month, per y^ear), is another component of instruciitmal time. Much 
research to date has focused on faciore that influence the pace of 
instruction, such as the organization and length of the textbot)k or the 
aptitudes of groups of students, and some <>f it su^ests that a more 
rapid pace is associated with higher levels of achievement (Barr 1987) 

Tfeachers in the Classroom Environment Study were asked before 
each observed lesson whether new content would be presented or pre 
vioasly taught content re»?iewed, In mcjst countries, the majority of the 
lessons focused on review of previously taught content, and, in mast of 
*hese countries, average gains between pre- and post-tests were mtxlest. 
Time spent on review, therefore, slows the pace trf iastruaion. If new 
content were presented every other lesson or every third les.son, mod- 
est gains in achievement could be expected. 

perhaps the component of time most frequently studied during the 
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past two decades has been student engaged time or timeon task. In the 
Classroom Environment Study, the correlation between timeon-task 
and student ^hievement was essentially zero in the vast majority of the 
countries. However, when differences in class abiUt>' and opponunity to 
learn were taken into consideratit^n. a weak but consistently positive 
relattonship between iime<in-task and stuctent achievement ^peai^. 
Apixirently il^ relationship oi timeon-task with student achievement is 
easily masked when txher, more potent, variables are included but not 
controlled in the study 

The major implicatton to be drawn from these findings is that edu- 
cators should resist the simplistic, linear, "more is better" view cf 
instructional and learning time thai has been so pervasive in recent 
years. Adding minutes to the schotii day, adding hours to the school year, 
and finding ways of keeping students "on-task" at all casts ire not likely 
to be a wise application of these findings. While adding years of study in 
a particular field is likely to pay dividends in stuttent achievement, one 
must add a sufficient numt^r of years. Based on the results of the study 
of French as a foreign language, Carroll (19' 5) concluded that six or 
seven years of study are needed to acquire satisfk-Kjry proficiency in a 
foreign language. Adding one nK)re year to an already skeletal foreign 
language curriculum is ni^t likely to pay off. 

Similar arguments can be made for pacing and homework. The 
results of the Cla«isroom Environment Study suggest that tlie pacing of 
nv^st teachers in most countries is quite slow, due in pan to the large 
number of review les.sons. While some increase in pacing seems appro- 
priate, one must avoid the tendency to move to the opfxjsite extreme. 
Too fiast a pace is likely to interfere with, not enhance, student achieve- 
ment (Barr 1987) The relationship between pacing and achievement is 
not likely to be linear: one more lesson per week devoted to new con- 
tent resulting in one more item correct on tlie end-of->ear achievement 
test. Nonetheless, achieving an optimal rate of pacing seems a wonhy 
goal. 

Similarly, to fecilitiue siudent achievement, homework tends to be 
superior to no homewtjrk. This finding does not jastify a call for more 
liomework (except in those circumstance.s in which homework i.s not 
being assigned)! Rather, the purptise and diificulty oi the homework 
(including length), and whether it is graded by the teacher (Walberg 
1984) ..lUst b«* addres.sed bt.bre this research finding can be translated 
into a stiund and defensible praaice. 
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Getwrnlizaimn 4 — D^erences in (eacbers* bebaiiors an urwekaed to 
i^ffere^K^ in sttddent achievement. 

An earlier version erf this gtmcralizaiion was more succina but 
inaccurate. It stated simply that teacter behavior was unrelated to stu- 
dent achievement. As our coll^ue, Michael Dunkin of the University 
trf Sydney, was quick to point out, this generalization is not in fea sup- 
|x>rted by tte data gathered during the Classroom Em ironment Study 
Th« ^udy focused on c^^^rences in teacher behavior, ratlter than 
teacher behavior se. The discinctton between these mo versions of 
the generalization is far more than semantic. 

The statement that teacher beha\ior is unrelated to student achieve- 
ment implies that what te^^rs do in their classrooms ck>es not influ- 
ence their students' achievement, tl^ teachers do not make a differeiKe 
in the academic lives of their students. Hits may or may not be the case; 
however, the lEA studies sted no light on this issue. Ratter, the Class- 
room Environment Study attempted to cteiermine whether differences 
in the behaviors mxed by observers were associated with differences in 
student achievement. Immedia^ly then, behavtors on which t^^hers 
varied little, if any were removed from consideration. If there were no 
differences among teachers on certain behaviors, they could not be 
related to differences in student achievement. 

As a consequence, two groups of behaviors were exchicfcd. The 
first consisted of behaviors exhibited fitirly frequendy by lar^^e numbers 
of teachers in vinually all countries. These included giving explanations 
to large groups of students, asking questions that required .student:; to 
recall what they had been u^ght, monitoring or supervising students as 
they worked at their desks or tables, and attending to housekeeping 
matters (e.g., taking attendance, distributing papers). 

The second group contained t^haviors rarely exhibited by any of 
the teachers in any of the countries. These consisi^ed of using examples 
to concretize an idea or abstr^k>n, asking questions thai require stu- 
dents to think or express and defend their opink>as, checking to see 
whetlK?r students understand what they are being told or taught, and 
stating that a student *s response to a question is incorrea or inadequate. 

The finit group of behaviors includes those tliat define the whole^ 
class, recitation-seatwork approach to classroom teaching. Apparently 
teachers throughout the world rely hea\ ily on this approach. The sec- 
ond group includes behaviors that hold promise for increasing student 
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achievement, but whase use in the classroom occurs so seldom that 
tiwir relationship to student ^hievemcnt remaias unknovm. Herein 
lies the fallacy of using correlational studies to presa ibe teaching prac- 
tices. 

In summary, then, teachers tend to rely on a limited number of 
behaviors as they seek to transmit knowlet^ and skills to their stu- 
dents. These behaviors, which for experienced leorhers appear to Ix; 
part of a well-defined repertoire or behaviors that they use as needed or 
desired, are apparently ct^nsistem with the current emphasis on whole^ 
class instrualon with the teacher assuming a directive role. 

Once these behaviors were exclucfed, we analyaxi the remaining 
categories further. During the Classroom Environment study, each 
teacher was observed for several lessons, typically eight. In one of the 
earliest stages of analysis, the consi.stency of teachers' behavior across 
these lesstjns was exammed. The results suggested that, despite the lim- 
ited number of behaviors olwcrved. teachers used them quite inconsi.st- 
ently from one lesson to the next. In fiaa, thi* inconsistency was so great 
that differences in behavior ctjuld be prediaed more reliably from dif- 
ferences in the lessoas than from diflenences in the teachers. This 
incoasistency implies either that teachers behave somewhat randomly 
from one lesst)n to the next or, more Hkely, that their behavior is consist- 
ent with the demands placed on them by each class of students, their 
subject matter or matters, the specific objectives they liave for their stu- 
dents, and their schtx)! and classroom oiganizaiion. 

The results of the Classroom Environment Study, then, suggest that 
teachers {!) do not differ at all on many of the behaviors thought to 
influence student achievement, and (2) do not differ reliably on otlK-r 
behaviors. As a consequence, tlx- finding that differeiKcs in teacher 
behavior are unrelatal to differences in student achievement should 
not be surprising. 

The Classroom Environment Study produced .some evidence that 
differences in teacher behavior are somewtet related to differencw; in 
student behavior, particularly in the extent to which studenus spend 
their classroom time on-task. Data on student behavior, like the data on 
teacher behavior, were collected during each lesson. Thas, clianges in 
teacher behavior were quite aseful in examining the relaiioashin 
between teacher behavit)r and student liehavior. In most countries, for 
example, teachers whv) spent more time interacting with their students 
during .seatwork and teachers who spent more lime giving explanations 
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Students who spent morv of their time on-iasW. TTiese findings make 
sense because studtents are likely U) take iheir fxrhavit)r cues from their 
teachers behavior (e.g., stay on-iask when teachers are physically near 
you or pay attention when they are expl^uninj? omething inijxmant to 
you)L 

TVo implicattoas can be drawn from these findings. First, and one 
that may not be greeted with cheers in light of the current American 
educational zet^^t, is that educators should stop kx>king for direct 
links between variation in teacter behavior and variatkjn in student 
achtevcment. Far too many faaor^ mitigate against such a direa rela- 
tionship At the same time, Ixnveven evidence supponing the relation- 
ship between teacher b:!havior and student tKhavior is more substantial 
and is consistent with the earlier finding that teachers tend to judge 
their effeaiveness mo^ on the basis of student behavior than student 
achte\'ement (Jackson 1V67). 

The second implication, c\)nsistent with the first, is that an exclu- 
sively behavioral view of leaching is potentially misleading and quite 
likely counterprcxiuaive. Jackst^n ( 1986) states his position on this issue 
quite succinctly 

There is ik> such thing as a behavioral definition of tc*aching and 
there never can be. We can never simply watch a pi^rson in action 
arid be sure tliat something called teaching is going on. . . . Our 
attempt to say when a person is or is m^t teaching is always an act 
of interpretation. We arv* forever "readers" of human aaion, seek- 
ing to determine which '^reading" is cnirea from among those 
possible (pp 77-7H) 

Tliree different "readers" of teachers actions were in classroi^^ms 
included in the t'lassrcxim Envimnment Study: tlx* observer, tlK' 
leucliL-r, ajid the studc-iil. Which iif lhc->e "leadcis" Ol\n'^de^ 'he nK)st 
accurate reading?' 

Genemiization 5 — Students ptraptiofis of ivfxit tmtispires in ttx' cUiss * 
room ufv tnon* lulkl than ifxx^^ of trained ohscnvn and, perixifis, 
teachers, 

Slightly m4)re than a dcx-ade ago, Doyle (197H) differemiatcxl 
among ihrcv jraradigms for research on teacher effectiveness: the pnK- 
ess-product paradigm (which posited a direct link betwc\.*n ic^acher 
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behavior and student achievement), the mediating paxress paradigm 
(which suggested that tlie ways in which students understand and act on 
teaching influence their achievement), and the classnxim ecology para- 
digm (which asserted that stutkmts attctnjH to negotiate a set t)f environ- 
mental demands as they seek to survive and prosi>er academically in the 
classroom) While the first paradigm gives the central role to tei^lK-rs. 
the other two give tl^ central role to students. 

Several of the findings o( the Classrcxim Envinmment Study sup- 
port the central role crfstudems in their own kerning. For example, siu 
cteits who see their classrtK>ms as Ixjsinesslike and academically 
focused tend to spend more irf their time on-task and learn more. This 
relatkmship was generally stronger than that between observers' esii 
mates trfihe percentage of students who were on-task and their achk.*ve- 
ment. 

Similarly, iliose students who refHm that their teachers provide the 
necessary structure for tlieir learning (e.g., informing them of the goals 
and objettivcs, fociising their attention on refcvant pt>inLs or features, 
or helping them understand the steps they siwuld foUow to arrive at 
correa answers or solutioas to problems) tend to have more positive 
attitudes toward the subject matter In contrast, none of the behavk>ral 
indicators associated with teacher structuring on tin? observation instru- 
ment were a>nsistently related to student achievement. 

Some educators may suij^^ that few implications can be drawn 
frc n this generalization since the stutk?nts' peacptions ratlicr than the 
teachers' belwvkjrs are more impiJrtant in determining student achieve- 
ment. After all. what can teachers do to influence student perceptions? 
We believe this to be a narrow interpretation of the generalization. 

Wliat this finding suggests to us is the neca f teachers tt) con- 
siandy mi>nitor their students efforts, to provide ftxxlback to students 
concerning the adequacy (or inadequacy) of their learning, and to work 
with them to correct their errors and misunderstandings. The role of 
feedback and correaives (Bkxim 1968, Rosenshine and Stevens 198()) 
in student kerning has long iKt-n rtxxjgnized. 

Teachers need to \x sensitive to .students' reactions to presenta- 
tions and explanations. The extent to which a teacher s explanation is 
clear depends primarily on the students, not the viewpoint of an exter- 
nal (^server Similarly, teachers must supervise students working at 
their desks or tables, point out errors as iliey (Kcur, prtwide suggestions 
for improvement, and, generally, enctjurage students to expend the 
lime and effon needed to complete the tisk successfully Just as obser\'- 



ers and students are exp^ed to "resid'' the behaviors of teachers, not- 
ing their intention and cx>asequences, teachei^ are exjxxted to "read" 
the behaviors of students, noting when and how to inter\'ene, arul wiien 
not to. 

Recognizing the centi^l role of students in their own learnlf^ diK*s 
not in any way diminish the importance of teachers. It simply redefines 
the teacher's role Instead of assuming that their presentations are clear 
and comprehensible, teachers need to verify this assumption with their 
stucfents. Instead of assuming that assignments are appropriate and lead 
to the ctesired learning, tc^nrhens need to nK^nitor their students work, 
providing feedback and correaives as netx-ssary. This generalization 
suggests that teachers should be less egocentric and more sensitive lo 
the impaa they have (or fail to liave) on their students. 



Conclusion 

Riwntly, Berliner 11987) propo.wd a "simple theon of classr<x)m 
instruciion" that includes the following as its primary variables: 

1. oppi>rttmity to learn 

2. students engagement in Ic^arning 

3. students* experience of success 

4. pacing 

5. siruauring 

6. monitoring 

lEA studies support including these variables in such a theory We 
suggest, however, tliat u much-needed emphasis on the central role of 
the students in their ovm learnuig is missing from Berliner's ihe<:>r\'. 
Each of the variables is aaually a student, not a te^^her, variable. Stu- 
dents need opponunities to learn tlxf knowledge and skills included on 
important tests. They need to be aaively engaged in learning, not pas 
sive. They need to experience a high rate erf suaess, not failure. Tlx* 
pace at which the iastruciion is delivered needs to be -^hptcd to their 
rate of learning — not too fast or too slow. Students need to know wliat is 
expeaed of them — what they are to do and what tht^ are to It*arn, 
Finally, the quality of completed work must be examirKxi. If stand rds 
are not met. students must be told of this faa and encouraged, wiin the 

r ; 
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teacher's guidance, to make nKKiificaiions so that the standards w ill Ix* 
achieved. 

Some educatoi^ might objt\i to the view of icai hing and learning 
suggested by Berliner and supponal b\ ihe results of the lEA studies, 
labeling it simplistic. We cone .. v^ ith Berliner, who said 

To such criticisms, I offer iwc> replies First, simple itkus ma\' have 
considerable utility: Seamd, fnim extensive uhser\ations in class 
rtx>ms, 1 liave learned that commonsen.se kntnvledge dtx-s ivn 
always result in either comnxin (tliat is, widespreatl) or sensible 
practices (pp 9'y-9<^). 

The strength of the findings of the lEA studies resides not in their 
lasights, but in their grounding in tlie everyda\ e.\|vrjcnivs of schixil 
administrators and teachers Krathwohl ( jyHS ) suggcsis Uiat researc h «n 
the Mx-ial and behavioral sciences, including education. Ikjs Hs greatesi 
impact when it allows us to 

compare the knowledge chum against our own eNpenencc- lor 
consisteiK-)'. Much beha\'ioral science knowkxls^e is te.sic-d against 
our own experience to see whether it " rings true" ipH). 

We contend that the knowledge piovidcxi h\ the llvV studies does, 
in fati, "ring true." The extent to A-hich ilv "i-onimoiisense" '<nowK-dge 
providtxi bv these studies becomes "commonplace" lenumis ihe 
responsibility of pratiitioner.v 
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The lEA Studies and 
Reforms in Reading, 
Writing, and Literaq'^ 

ALANC. PUKVES 

Tiiroughout its history, lEA has i ondiKUxl ihrec* siudivs oUtiucd- 
itui. in the mother longuc; onv in a-ading, one in liicraiure, 
and one in written CDniptisitiDn Studies of foreign langu;ige 
learning (French and English) wea' also condiKic-d, but I sliall ni « aner 
those here. The first two surveys were completed in the e;jrlv 1970s and 
the third in the mid-1980s. Their results oflfcr ways tt) use coni]iaratjve 
international data for chiming the direction of curriculum reform 



The lEA Reading Study 

The Reading Comprehcnsitm Stud> (TlK)rndike ly"'^, Purves et al. 
1980) used a straightforward test of amiprehensiivn of passages, fol- 
lowed by multiplv-^choice ijuestions, plus reading speed anc! word 
knowledge tests. The instrument was administereti to thrtv ]X)puIa- 
tions; IC year-olds, 14-year-olds, and students in the prt^universitv pro- 
gram, 

The twerall results shtjw that United States students }xilbrmed 
about as wtll as the students in most industiiali/.c-d countries, 1-ut that 
New Zealand students stcK)u out on thv ctimprehension measure abo\ e 
all others. The various initial anah-seH n( the reading compa-ticnsion 
data pointed clearly to the influence on aclue\cnieni of itie Ikmi*: hack- 



Alan C rum's Ls imxtor of tix' Cvutet jor \ii-itiH^ and LitemcY ctnU 
Pn>fisimr uf EdiuMtion ami iiiwumitu'ii, JU' I ntix'nity of A/fmiy, State 
Vnii vmit}' of Ncu • }brk. 




Reforms in Reading, WRmNt;, aw Literacy 



ground variables, particularly ht)inc literal' variables. This influence 
was i^parent both within and between txjuntries. 

Although these analyses indicated little trf cxwsequence for scluwl 
reform, they prompted a number erf subsequent studies, particularly 
those of Purves 1973, Purves et al., 198(), Baten 1981, and Guthrie 1981. 
The results of some of these studies supported lEAs initial analysis and 
sufsi^ied little could be done to raise a system's reading scores save a 
massive restructuring of stxriety to make print literacy more predtimi- 
nani in the culture than it is, and that such an effort, even if possible, 
would fail because of the influence t)l electronic media. At the lime 
testing, New Zealand was, relatively sptiiking. televjsi'.>n free and heav- 
ily print oriented. That is no longer the case only six years after the ini- 
tial test. 

A more promising line of inquirv for educational reform was to 
reexamine the passages and test items (Purves et al. 1980, Baten 1984X 
The results of this inquiry sh<)W that, for the 14-year-old population in 
most countries, certain passages are .systematicallv more difTicult. Their 
difficulty, hcTwever, can not be explained by the usual fomiulas, such as 
readability, but it can wlien the structural organization of a passage is 
amsidered. 

For this age grt)up, passages with other than a sequential or narra- 
tive struaure are systematically more difficult. Pas-sages with cause^and- 
effect or comparLstin-contrasi struaures are seldom available in school 
texts through junior high. These texts see much more frequent use in 
upper .secondary sch<X)ls and form the Ixisis of academic writing in 
most of the industrialized and developing nations. Students with little 
previous experience of them appear to have great trouble reading them 
on a test. 

The analv.sis of item types .shows a similar re.sult. Of the four 
types— meaning of parts, meaning of the whole passage, inference 
beyond the text, and analysis of rhetorical or struaural devices— the 
fourth proved to be much more difficult for 14-year-olds in many of the 
countries in the study'. Analysis t)f the textb<>oks and the curriculum 
show that, although questions ol rhetorical device and .struaural analy- 
sis appeared in many .standardized tests, they were not pan of the 
instructioi/al program for students until upr^r-secondar>' .schtiol. Again 
the students performed fxxjrly l?cn:ause they did not have the opportu- 
nity to learn what was covered on the lest, even though testmakers in 
many of the systems said that these were appropriate items and passages 
for the age group. 



4^ 



llie findings of tlii.s siudy have since btx'n suf^ported by flie wtjrk 
of many researdicrs in iltv llniled States and elsewlierc and arc 
included in the vulumc lifcontiftfi a Nation c/ Readers. Clearly ilie 
implications for currii ular and fxxlagogiral reform are preM.-nt here, 
but whether tcxtlxxiks used in the United States (especially the basal 
readers) have been revised to include nonnarrative material and tjues- 
tions that deal with rlunoric and structure has not IxvonK- evident even 
in the more recent editions. 



The lEA Literature Study 

The lEA Study of Literature (Pur\'es 1973) conducted in the early 
197()s invx)lved 10 cviueational systems. Flemish speakinj^ and French- 
speaking Belgium, Chile, England, Finland, Iran. Italy, New Zealand, 
Sweden, and the United States, llie result was similar to thai of the read 
ing study Again students frt>ni the llnitcxl States were wvll within the 
average, and the be.st student.s in tlx? last >'ear of secondar\- schixil out- 
iTerftirmed ilie students in all oilier ctjuntries 

Of the variety of literary works presented to the students, liowever, 
one provtxl much more difficult than the others. If was the one elliptical 
and Jitetaphoric .story, written in a style thai might Ix' termed pt>etic 
rather than literal. Examination of the iexthtx)ks slxiws that this sort of 
writing was virtually absent from the fare presented to students, particu- 
larly in the llnittxi states, through .secondary sclxxil. 

The analysis also shtms diat students, particularly at the H-xear-old 
lc\'el, did much more pixjrly on items that asked for tlx« application of 
literary terms tt) the material than they did on items dealing with inter- 
pretation, cither literal or infea^ntial. lext analysis and data from teach- 
ers show that liteniry terminology is not triughi th.-se students e\'en 
though it is considered an appropriate part of the curriculum. As in the 
reading study, the literature study shows that opportunity H) learn is an 
imponant factor in student achievenx-nt and is clearly an area where 
educational a-fc^rm can ixcur 

In a accent reanalysis of tests in reading and literature in the United 
States (Bnxiy. IX-Milo, and Punes 19H8), unftiriunately little change 
apjx-ars in test items and passag'.'s loonnarrative and figurative passaged 
arc .still lacking, and jtem.s dealing with the application of anahtic terms 
and concepts from rhettjric ar literar>' .stud\' are in the mitxjfity A cur 
.sory kK)k ai tlx* textbiH)ks again shows that the majority of texts contain 
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literal nan^ives and the terms used for analyzing style and srruaurc 
are myt en^hasii^l in the teaching learning apparatus sum^unding the 
texts Opportunity- to learn these more complex aspects of writing is 
still nonexistent for nK>st stuttents in the United Stmes. 

Hie literature study also examined the preferred respoa*^^ of stu- 
cfents to literary texti? (eg , what students think is important to write 
about) On the basis of a content analysis of e:K:h senteiKe in comp^...i 
tions written by over 1000 stuctents and teaclu^rs in it!sponse to a variety 
of shon stories and poems (Purves and Rippere 1968), 20 core issues 
eT^rged concerning literary texts ihm could be framed as questions. A 
larger sample of stuctents in each a)untry were then asked to selea the 
five erf these questicMis that they thougiu mt^t important to raise in writ- 
ing or talking about a short story, tach stuctem made three respoases— 
one cxh folkwing the reading erf two stories and one in a question 
naire abou: fution in general ¥mm these we derived a picture trf what a 
student sees as important in the abstraa and after reading each (rftwo 
stories. The questioas were rated by their teacfKrrs and by a group erf 
experts on literature and its j^d^^c^ in each c\)untry 

The results show that, as students progress through secondary 
school, they become more coasistent in their responses across texts and 
thus come to approach all the fiction they read from a single pK^rsf^- 
tive. Students in any one country tend to have increasingly similar 
i-espoases as they prqgress through school This is not to say that the dif- 
ferent stories h2ul no effea on the students' responses, but that in each 
of the systems of education studied, the effea of the text was less stn>ng 
on the older students than it was on the younger ones. 

In terms of national diffenences, the results show that there are 
both an imem^ional interpretive community and national interpretive 
communities. Students in all countries generall>^ rejeaed the questioas 
*1s this a proper subject for a story?" and "Is anyone in this story like the 
people you know?" They generally found attractive tlie questions lias 
anything in this story a hidden meaning?" and "Wnat happt^ns in the 
story?" 

Aside frt^m these general rejeaions and predileaions, students in 
different countries exhibit sharply different profiles of response?. 1\vo 
sets erf questions form the cex)rdlnates on which we pknted the major 
tlifferences between and among countries (Figure 7.1 ). They are, first a 
'^personal impersonal'' continuum and, second, a * form-ce>ntent" con- 
tinuum. Students in Belgium and Italy emphasize the impersonal and 
tlte formal; in Chile, England, and Iran, the personal and the content ori- 
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cnied response; and United Slates students are concerned with a)r.:em, 
but not from a personal point of view. 

Responses to reading tlwn, although panially dependent on the 
particular text read, appear more crucially dependent on the culture of 
the reader, the reatfer's membei-ship in a community of readers. Such 
dependency increases as people progress funher through their educa- 
tional sy.sieuj. Students' responses become increasingly like the 
responses preferred by teachers and, to a lesser extent, experts (critics, 
curriculum makers, and tearher trainers whom we polled), For the 
younger students in the lEA Literature Stu iy (age 14X the average rank- 
order correlation between the queaion preferred by student and 
teaclier was .35, for the older students (in the pre-universiiy year) it was 
.54. Hie teacher, therefore, ser\'es both as a nr>a|or force in influencing 
how siuctents respond to a text and as the individual w1k> inducts stu- 
dents into the interpretive ctwimunity of a panicular .society. 

The question this result r«dses for educational reform is whether 
the limittxJ and rigid panern of respt^nse that students learn Ls the most 
appropriate or desirabk? to take into adult life. Reform documents call 
for critical reading and varied ^^proaclKfs. United States students learn 
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that eac1i Htcrar^^ text has a hidtkm symbolic meaning containing a 
HK^ral or lesson. Such an approach to re-adinp has ccHisequences iKnxmd 
tlK* literature curriculum, making it difficult for iK'ople ttx) explore 
reading material that d<K*s mn fit tlK* limitations of tlx- pattern. At an 
extreme, it IcMds to censorship and lx>t>k burning. Is thai truly the goal 
of tlK^ literature curriculum of American schixils? 



The lEA Written Conqxisltion Study 

Hie IK A Study of Written C:om|>ositicjn currently invo^%^es siuckfnts 
w1k> were tester! in 198S in tl>e foHowing countries: Chile, England, 
liamliurg (Tlie Fcxk-ral RepubUc of Ciermany), Finland, Hungary. Indo 
nesia, Italy, Netherlands, Nigeria, Neu^ Zealand, SwtnJen, Thailand, tlx* 
IJnitetl States, and Wales. Three |X)pulations participateii. students at the 
end of compulsory educatum, roughly tlK' agt^ of 15 to 17 and the main 
fcx'us of this study; students at the end of primary cnJucaiion (grade 12); 
and students at the end of tlie prc^utiiversity year Ikxause i\yc results are 
still being analyzed, comments made at this {xnnt may be mixlitied by 
future findings 

The students wrole on parallel Uisks and the scoring useil a com- 
mon scIufttK- and scale across languages and cultures. As a a^suli, cer 
tain cijmparisons can be made but it is im}X>ssible to m;ike a dircvt t urn 
parison of the writing performance of strdenLs in different countries, in 
p;m bec ause of language differences and in pan Ixxause of the proh- 
lems of scoring com}>i)sifit>ns internationally 

In the main pi^wlation (ages 151 7), each student wrote on three 
tasks, one a functional task, otie a descriptiveexj>ositor>' letter, and one 
either an argument, a narrative, or a reflective essay The relative pcT- 
formamv across tasks wxs nut unift>rm, Alfhougfi tlx? narrative was i\yc 
easiest and argument or reflective ess;iy the most difficult in eleven 
of the systems, this nnus ncH the case universally In three systems, the 
reflex live or argium-niutive task was relatively easy Tliese were the sys- 
tems in which such tyjx's of com}X)sitit)n had replaced ilte narrative as 
the dominant form. 

Anoilier initial finding shows that, in systems with streaming or 
tracking, students in lower tracks, as might lx» expectexl, come from 
homes where literac\ practices are few. hi class, tf>e\ write less and 
receive less instruction in writing. These* s^udent^ tend to penreive suc- 
cess in writing in terms of handwriting, neatness, spelling, and granv 
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mar raiJkjr than structure, .sr\'k% or cotitent. Such a result, combined 
tlK- first, clearly suggests ihv imix>rtance of btnb ciassrtxmi oppor- 
tunity to learn and the curriculum to ensure that tlwy have a chance it) 
learn and pr^ice what the>' are measured on One can attack teaching 
to the test, hut shouldn't tnie also attack testing the untaught? 

Another asfxxt of the study is exploring iIk- extent to which stu- 
dtnts are taught to he members of rhetorical ctimmuniiies. It is examin- 
ing th- criteria used by teacliers of wriung to see if there are systematic 
diffciences that might lielp define rhetorkal communities, and it is 
examining the perceptions students about what constitutes "giKxi" 
.school writing. 

The hypothesis underlying this exploration is that students learn to 
write according to ix-rtain conventk)ns, many of winch have liitk- to do 
with the structure t)f tlx- Lmguage and more to do with tJie literary and 
cultural heriiage of the society. That is to say. many aspixts of texts are 
mn bound by the morphok^- and grammar of a particular language, 
but by custom and convention. Such is the li:K' of a-asoning Ix-hind 
wtitmsiii V rf.H'tork. 

As an initial step, wegatlH'red a number of comi?osiik)ns written by 
^pn)xim-itely 100 students at or near the end of .sixondary siacx)} in 
several of the ctuiniries in the study and three countries no longer in 
the study (Australia, Japan, and Israel). TIk- compositions were on "My 
Naiive Ttwn" and "What is a Friemi?", bt)ih tl7t)ught to be a-Iatively neu- 
tral u>pics that wt)ulLl not force a panicular pattern of oi^ani/'.ation or 
style on the students A team of three reseaalK'rs then examinetl ilie 
comfX)sjtions and found they tended to differ systematically h\' culture 
of origin along a number of continuums, some of which matchetl those 
of earlier researchers, particularly Carroll (]%()). Kaplan ( 1%6), (,lenn 
and (Jlenn ( 1981 ), and Hofstede ( 1980). They are the folk)wing; 

PEIiSONAl. IMPERSONAI. defx-nds primarih on liie frequt •no* nf refer - 
ences to tht- writer's thoughts and feelings alxnit the subjui. 

ORNAMEN TKI) Fl^IN may also Ix- defined as ■■figurative literal ' and 
depemls on the frtxjuency of use of metaphor, imagery, and other fig- 
ures of speech. 

ABSTRA(rr-i;ONt:RETE refers to the amount of s|x*cific- inftirniaiion and 
detailed references in the text, as well as to the general level «>f abstrac- 
tu>n. 
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S!NCjI-E MIUJIPIJ^ refers lo wlKthcr tlie lext fcKUSi^ un onv asixvi uf 
the subjtvi or rries lo cxwvr a large numlx-r 

PRC^POSITIONAL APPOSITIONAL Thijs continuum, which is similar lo 
Glenn and CJlennV abstractive^assix'jauve aiul some itf Kaplan's dia- 
grams, Offers to iIk* tvfx^s erf conjHxiives thai hold the lext together and 
wliellxT the com{X)sitiun is devek>peii by a standaai order. A conipari 
,son<onirast coniposiiion, for example, uxnild he pnjpositional. An 
t)ppositional composition uses few conneaives besides a^it/ or hat and 
irfttTj omits ct>hesive lies oiIht than idioms and re|>etiiions TJie contin- 
uum friim ap}x>sitional to prt){'H>siuonal is one ffiai moves from stream 
of cons4'iousnc*ss thunigh narration and description to tlx* syllogistic 
prcKif 

These characteristics ap|x*ar to disiinguisfi the writing of students 
in different cultures and to form sonx* of the dinx*nsioas tlK^t dehmit 
mtxlels of lext in cert;un cultures (see Figure 7.2), These differences are 
not inluTeni in ihe language, but result from cultural k-arning ^xi ause 
ihey occur between students writing in the same language (such ;is 
English) but living in different cultures. Because full analysis has mx yei 
lx!en completed on tlx main btxly of comfH)sitions, tlx*se comments 
must be seen as suggestive latlx^r than d^'finiiive. 

In tlx* pilot phase of the study leachers fnsm tlx particii-uting 
Ci^untries were asked to Ixnh mv and comnx^ni on a number of the 
comjx>sitions. Their comments were analyzed for content, followcxl by a 
factor analysis. Four "geiu^ral merit factors** enxTgt*d apart from 
mechanics, s|x*lltng, and handwriting: (1 ) content, (2) i)rgani/iajon and 
structure. (3) style and tone, and (4) pcTsonal resj-nmse to the writer 
and the content (Purves 19H4). These factors of judgment apjXMred in 
all i*oimiries, but tlx- relative t*mphasis and interpretation varietl from 
system lo sy.stem and fron\ tyjx" to type of com{>osition. 

To chetk tlx- teaclx*rs criteria, students were asketi It) write a letter 
of advice to people vxnmger than they who were al>out to attend then* 
S4hiK)ls The letter was to suggest ways to succeed in schix)} writing, and 
content analysis of the resulting com|X)silions provc\l most revealing, 
:ilthough nt)t necessarily a)mplimentary to the schtx>ls Stutlents in all 
the countries nott*d that spi^lling, handuTiting. grammar and ixatixss 
were of paramount importance. 

The letters were also analyzed for specific tvfxs i)f ad\'ice, and it 
was divided into the broad categi)ries of content, organi/Hition, style and 
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lonc (following ihe si'oring scIkmiu ), and prcscniaium, iiuluiling gninv 
mar, onlwgraphy, lavxiut and innunt^s, pnxx^ss and classuHini taaics 
(Takala 1983). 

Ailhough ilie analj-sis for all tounirics is n<n aimpletc, ihc results 
from a few of ihcm strongly suggest that students within a a>untr>' arc 
aware of iIk* existence of certain operating norms Variatii^n ixxiirs 
within a country; of cxiurse, luit specific panerns exist nationally and 
internationally Figure 7,3 identifies tlK^se patterns 

This figure reveals the general perceptit)n of students that tluMr 
ieachR?rs are pretxrupied with matters of prc*sentatic)n. Tliere are strong 
national differences in jx^avption. htAvever, such as the relatively km 
efpphasis on organization in c;hile and on style and ti)ne in Nether 
lands. In New Zt*aland and Swt\len, te;ichers appear to enipliasize pnx' 
CSS more than in ot^ier countries, but m Sweden nK>a* of this emphasis 
a)ncx*rns choice of topic than is true ir New Zealand. DiffereiKt^s cxcur 
within ottier cm'g<>ries as wvIL In The Netherlands, the emphasis under 
presentation is weighted! more towards gramipar than in the other 
Ci)untries, wIuTeas in Chile, tlie balance tips tH?avily toward appearance 
and spelling. Under organization, a strong a>ncern is shown in The 
Netherlands for using an organization set by the teachers, an issue of 
less imponance in the other a^untries. In Swcxien, leadK-rs seem to 
favor simple seniencvs, a feature that does nu figure much in the other 
countries. Fn>m this evidence, ir appears, acxx^rding to ttor students, 
that teachers favor particular ways of preparing ;ind presenting written 
discourse, 

Riich of the three asptxis of the Written (A)mposiiinn Study (ihe 
examination of com{X)sitions, teacher questionnaires concerning crite- 



Figure 7.3 
Summary Peicentage Distributions 
of Student Advico in Five Countries 
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ria, ami the leucr of .klvkv) sii^j^cmn lhat suuk-nis in An LxliKAUnml sys 
lem di> i!ukx\i learn to btvomc nivmbcrs of a rlK'U>rical iunimiiniiy, 
that ilicy learn not only iitnv lo write, hui also w hat aspivts t)f their writ' 
ing are vahied in ilwir sixiviy At times, there is dirixi insirunion, at 
linvs. it is iniphat, but eitln-r way, the srudenis le *rn that lyvinn lo 
write is bein« able to pnxluce texts that niatdi ecTUtin nuKiels ai.tl that 
tfx'se nnxlels serve ;is stuck«ni and !e;Kher c riteria It uv pur tlK-sc find 
ings together wiih the findings of ihe literature Mud>. v^v can scv that 
learning literao' i i a sot ieiy is not merely a matter of aaiuiring a set of 
skills. The implications }or reform are clear Hw c\lucatu»nal community 
slH)uId determine wliether ilw imnlels of text and t»f reading and writ- 
ing are what is desircxl aiui tfkrn the extent to \shjcJi the testing and 
teaching programs (including te;H'her leixKxak on student writing) 
ainftirce or undermine these goals 
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Science, Mathematics, and 
National Wel&re: 
RetrospectKre and 

Prospective Achievements 

HERBERT J WALBERG 

In The Vilify of Natiom, Adain Smith (1776) ai^ucd that national 
welfare depends on ability. Given the impt)rtance of this dien- 
assumed dependence, the direct evicfc-nce for it seems slight. In A 
Natmn ai Risk, for example, the National Commission for Excellence m 
Education (1983) warned that educational mediocrity jeopaixlizes US 
prospects for growth in economic prcKiuaivity and welfare, but cited no 
correlatioas of education and economic data. In view of current public 
and legislative wncem, this study explores die relations of education 
and economic indicaium 

Economic indicators arc hardly the only criteria for evaluating the 
quality of life or nationa' welfare. Income, however, substantially deter- 
mines tlie amount ind quality ttf domtsiic and foreign goods and ser- 
vice> that petjple can buy Moreover, e\'en small differences in sustained 
economic grcTwth caase large differences in national standings. 

From 1870 to 1950, for example, as scIkx)! and university enroU- 
menLs expanded in percentages of eligible age groups in the U.S., the 
gross national prtxiua per woriccr grew 0.7 percent faster than that of 
Europe as a wlxiie. This small advantage in growth sufficed to place the 
ILS. first in per cq?ita income and economic growth by mid-century 
(Schjirz mil ' 
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As rc'ixMifd below, grimh in iiKDnif differs widely anu)ng naiions. 
Such differences, it su«aiiKxl. will cause vasi changes in nanr.nal wealth 
ranking*^ At 1977-1982 U.S. and European ^rtTWth raicii oi around 1.5 
percent, for example, income would take 47 years to double. At the Jaf)a- 
nese rate of 4.4, it would take 16 \vars, Tliese figures may undere.stiniate 
ultimate diffea-nces since Japan's ina>me now exccxxls that of the U.S. 
a'-'l was grtnving at tlie rate of 1 1 .3 percent in the first quarter ol 19HH 
. bA 1988, p 29). 

Education Indicators 

In txinsidering htnv tt> measure the |X>ssibIe educaiitin-growth con 
ntxiion, what slxHild be clxxsen as education imlicators is immedi- 
ately obvitnis NaiiiMial agencies (ft)r example, the U.S. IX-panment of 
Education 1988) report quantitative indexes trf input, prwess, and out- 
come such as per-student expenditures, enrollments of age-eligible 
groups, and mean years of eiiucation in alult |>opulatit)ns. Yet tlie 
National Qimmissicn, other reform groups, and legislators want quality 
indicators such as results on achievement tests. 

Such insi.stence may tm be wiilwui merit. Per-.student expendi- 
tures, for example, appear unlinked to ;K'hievcment gains in an ynalytic 
review of more than ISO studies (Hanushek 1986). Moreover despite 
jxx>r science and mathematics performance, the U.S. recx'nily spent a 
greater fraction of gross national prtxluct and of total government 
exfA'nditures on education than many other countries At 18 percent, 
the U.S. spent a larger portion of its total tax dollars on etlucation than 
any other ctnmirv' recently survexed by he United Nations (U.S IX-part 
ment of Education 19}^, pp 16-17, 3S). Indexed, considerably more than 
the second and ihirtl ranked tx)untiies, Norway and Sweden. 

Nor can enrollments and average adult years of c>ducation be t;iken 
as valid indicators of results In 240 days per schtxil year { National Com- 
mission 1983), for example, Japanese students earn in 12 years the time 
equivalent of 16 years t)f U.S. students' 180 day year. This is aside from 
other pt)ssible Japanese advantages such as a rigorous national curricu- 
lum and educatijmal encouragement at home (U S, Department of Edu- 
cation 1987, p. 79). 

In view t^the $850 billion cxiucaiional investment embtxlted in the 
U.S. labor fon from 1900 to 1970 (Schultz 1981). it seen^s irtmic that 
natioas have such spotty and noncomparable indicators o! :iuman capi- 
tal in contrast to the many well-standardized indexes of labo. and physi- 
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cal and financial capital — many available in modern amntrjes mi>nth]y 
over decactes (see Becker 1976 and Mincer 1981, on the coiKepr ctf 
human capitalX Several national and iniemattonal organizations, htnv- 
ever, are beginning to coltett comparable information Dver time and 
place. These inclucte the Natfcnal Assessment of Educational Progress, 
similar programs in other ccftintries, and the International Association 
for the Evaluation Educational Achievement (IEA> 

The Paris-based Organization for Ecx»nomic and Cooperative 
Develc^ment, moreover, inaugurated a l^gescale indicator program in 
mid-1988 thai will enable scholars and politTmakers to compare the 
progress of countries with respea to educational inputs, processes, and 
outcomes (U.S. Depanment of Slate 1988). Thus, we will he able within 
a few years to relate educational investn^nts and prwrtices tt) accom- 
plishments in the hope of improving policies and investigate tlie possi- 
ble contributions of accomplishments to national income and welfare to 
suggest allocatioas of educational im'estmenLs. 

In the meantime, this study explores the second possibility by relat 
ing indicators of educational outeomes to growth in national income 
and other welfare or quality-of-life indicators. Ttese data were compiled 
from indicators collected by staff researchers of the Qxmdon) Eainomist 
(1983) and in three lEA studies of science and mailiematics achieve- 
ment (Comber and Kccves 1973; McKnighi, Cras.swhitc, Do^y, Kifer, 
Swafftird, Travers, and Ccxiney 1987; and lEA 1988). 



Level and Growth of Income 

After World War II, tlie U.S. was first in natif^^al income and eco- 
nomic prowess, but other countries caught up and surpassed us. As late 
as 1965, for example. General Motors earned twice as much as the com- 
l)ined totals of the ic^p 30 German and top 30 Japanese industrial compa- 
nies, but in 1^ GM reported its first deficit since the Great Depression 
and in 1982 announced that it would import and sell Japanese cars. 
From 1965 to 1980, the U.S. share ctf cx>mpanies with world sales over $1 
billion declined from 70 to 40 percent (Freund 1982) 

By 1^1, as shown in Figure 8.1, the U.S. slipped to fourth place in 
per c^lta income and ranked in the bixtom half in economic ffowih 
among 16 nattons with comparable data. By 1988, the dollar had Imi 
substantial value in relation to other currencies; therefore, the current 
US. rank is now conslderaWy lower. 

o lU 

ERIC 101 



Sc«?NCE, Mathi?matk:s, and Natk)Nai Wfi^fark 



Figures 8,1 and 8.2 shew that high-ina>iw countries ga'w shiwly. 
The low-iiKx^mc countries— Brazil, Hungar>; and Mexico— averaged 4,3 
percent growth, and the high-incoMC countries— Switzerland, Sweden, 
and Germany— averaged only t^)ut 1.5 percent. Simple regression 
shows that each additional $1,000 in iiKx>me is associateti with an aver- 
age drop erf 0,25 percent in grt>wth (see Figure 8.2 for bivariate correla- 
tioas and significance levels) 

The important exceptions are the Atlantic nations — Britain and 
Spriin — with relatively inferior iiKome ami growth (below the fitted 
regression line) and the Pacific nations— Japan and Sin^x)re™which 
grew much faster than their incomes would prcdia, Japan remains 
unusual since its recent growth has been extremei/ high, 11 percent, 
despite being first or near fin>t in per-capita irKX>me sii ce the rwfar dou- 
bling of its national currency in relation to the dollar 

As Kennedy (1%7) ai|!ued in his widely mH<xi history, Tf^Riseaud 
Fall the Great Powers: 15002000, tlie mighty Western nations (Spain, 
ftmugal, Holland, England, France, Germany, and the U.S.) have risen 
and fellen in wealth and power in relation to one anotlier. Desjiite its 
economic ascendanc>' in the fxrsi half of the century, the U S, in 1981 was 
26 percent below Switxedand in per<^ita income and 64 percent 
below Japan in economic growth (Figure 8,1 ). What faaors amount for 
economii growth and national w clfare? 

Creative Elites? 

Among philosopiiers who auribute primacy to elite aristixrnicies or 
oligarchies, Plato believed that philosopher-kings brought out the best 
in a Greek ciry-state. Among historians of the rise and fall of civiliza- 
tioas, Arnold Toynbee (1959) held a "great person" theory erf national 
devetopment. In his voluminous A Sttuiy of ffiston^, To>iibee hypothe- 
sizcxl that national ascendancy is the result the enKTgence of a "crea- 
tive minority'* that leads the masses to greatness 

One possible world index erf such leadership in modern times is 
the Nobel Pri?.e, given in recognition of outstanding kradership in the 
natural sciences, economics, medicine, literature, and the pursuit of 
peace, Ho\n t^ver, the ass(x:!ation between Nobel prizes and eainomic 
growth contradias the great-person theory As shown in Figure 8, 1, the 
nations that have cumulated th^: highest number (rf Nobel prizes per 
10 million population — Sw^eden, Switzerland, and Britain — gr*^^' 



Figure 8.1 

Economic, Educational, and Culturai indexes of 16 Nations 
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Grass dm>eslic product (GCP) pef capita, 1981 

GDP amage amiual gr{Mth rate percent, 1977-82 

Unempk^ed percent of lalx)f force. 1 962 

Number of prizes per 10 million |X)pulatiOT, 1901-1982 

Percent of 20- to 24-year-olds enrdled in higher edicdtion, 1979 

Sconce test scores of 14-year-o)dF. 1970-71 

Avenge f^HJrs worked per week (nonagricultufal). 1982 

Ftercenl dt marriages ending in divorce, 1979 

Number of television sets per 100 populatiw, 1980 



Afcrfe. On^y 9 of the 1 6 countries participated in the 1 970-71 science sur^ of the Inter naiK»ial Associaton for the Evaluation of EducatKxial Achievement. 
Soufoes; £cono?77^sr. December 24. 19a3. Walb^ 1983 



ERLC 



11 



Figure 8.2 

Correlations of Economic, Educational, and CuKurat Indicators 
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skwly, and tlK)sc with few prL/es— Singapore, Mexico, Brazil and 
Japan — grew mo^ rapidly. 

The negittive relation, earlier noied by Wallx^rg (19H3X has been 
exhaustively replicated with Ncrfx"! science and medicine prizes, inathe- 
maiioi prizes, and publication and citation raic^ for varii^us periiKts by 
the Congressional Research Sei^ia^ (CRS 1986) for thv^ I LS. House Com 
mittee on Science and Technology. Tlxxse whK) advocate si>ending for 
science to promote economic growth may overv^alue discover); under- 
value ux:hnok)gy and application, and fail to recognize that scientific 
information is ra|>idly disseminated around the world. If human capital 
is scarce, moreover, funds expended on disa)ver>' may not exploit it 
(Walberg 1983, CRS 1986). 

Hafdwotking Elites? 

Another index of the effea erf the el«te on a nation is the percentage 
of \x>ung adults enrolled iacollert\s and universities. This indicator also 
predias slow ecc^nomic growth figures 8.1 and 8.2). 

Each percentage point of additional enrollment was asscKiatcxl 
with a 0.1 jH^rcent drop in grovvih. Acn>ss the range c^f college enroll- 
ments, an increase from 3 percent in Brazil to 55 percent in the U.S. was 
associated with an estimated reduaion in economic growth [rom about 
7 to 1 f^rcent. Tl^ U.S. had by far the highest fraaion of 20 to 24 year- 
olds enrolled. Canada a!!d Sweden also had high college enrollments 
and airrespondingly low grcmth rates. 

Larger enrollments in higher education, slK)wn in the figures, are 
also associated with a shorter wori< week. Each additional jiercentiige 
point in enrollment equals 0.3 hours (18 minutes) less work {x?r week. 
Acrass the range of enR)llmems, fn)m 3 percent in Brazil to 5S {learnl 
in the LIS., this amounts to an estimated 15-hour reduaion in the worii 
week from dhoui 47 to 32 hours. 

In the countries with high cx>ncentrations of c<)lleg4>educattxi 
adults* how is leisure time spent? a>Uege enrol Imetits predia high rates 
of television ownership, up to 62 per 100 fXTSons in the IIS. at tl>e time 
the data were ailfecied. 

Finally with respect to cultural correlates of enrollments, another 
indicator was significant: high college enrollments are assc}ciaied with 
higli rates of divorce per I(X) marriages, up to 50 and 53 in the ILS. and 
Sweden- 
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Achievement Ttesf Scores 

As shcmTi in Figures 8,1 and 8,2, the scores H-year-olds on the 
standarcUzed lEA erf' Scimce Achievement collected in 1970-71 predicted 
economic growth a decacfe later Each one^point rise in the averse test 
score is assiKiated with 0.2 percent higher growth 

Among live eight nations that panicipated in the lEA » n Science 
Achievement, Japan h^Ki by far the hig^^ score, 31, and alst^ r^»c highest 
growth rate, 4 A, Italy, Britain, Sweden, and the U.S, had scores below 23 
and correspt>ndingly low ea)nomic growth below 2,5 percent, 

High test scores, moreover, predia low rates erf unemployment. A 
one^point rLse in the national average test score was associated with 07 
percent lower unemployment. Jiq^an is again noiaWe in having a low 
ur^mploymeni rate (2.4 percent) while Western nations with low 
scores— Italy, Britain, Sweden, and the D.S.— had high unemptoyment 
rates (5.1 to 12.5 percent). 

Thus, natii>nal affluence and growth are demonstrably intertwined 
with indexes ctf education ami culture. TTie rankings seem to indicate 
impt>rtant characteristics erf natioas— their present ami future. It is diffi- 
cult, nonetheless, to determine iheir causal relations, value, and implica 
tioas. Hisioriaas and social scientisLs have reached little conseasus on 
why natioas wax and wane. Reasonable pet^ple may also differ in bow 
they interpret and evaluate the various indicators and their causes. 

Yet it is important to try to understand these phem^mena, Educa- 
tors, in panicular, should be concerned with the possiWe influence cf 
enrollments and test scores on national welfare. How changes in educa- 
tion might affect our .society and ecorK>a/ is wonh thinking about. 

The Inforniatton Ecxmomy 

Moctern life is increasingly concerned with the generaifon and 
processing trf information. This prospect seems likely to demand an 
increasingly autonomous, cooperative, intelligent, and nK)tivated work 
force (Walberg 1983). Agriculture, mining, and heavy manufkiuring 
occupy fewer of as today than in the past, and high \^ue^added manu- 
facturing such as electronics and pharmaceutic's and services siKh as 
insurance and data processing are growing throughout modern 
natfoas, 

Altlwugh scientifk and engineering breakthroughs occasionally 
raise produaivity of these industries dramatically, prc^ess comes 
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mainly from adapting already-discovered ideas. These coasist «if small, 
incremental improvements in aaivities, materials, and comi^mente; 
recfesign or substitution of .^iviiies. aJmponents, and pnxiucts; reduc- 
tions in costs; and advances in performance, quality, style, and am- 
sumer appeal. 

lb discover, plan, implement, and measure the results of such 
changes in traditional or high technology requires manufacturing, ser- 
vice, and sales forc» who are knowlediseaWe and motivated. It also 
requires competent workeni— not necessarily superior scientists, man- 
agers, and other leaders—skilled with materials, technotagy, and quan- 
titative data. They also have to be able to absorb and propagate knowl- 
edge as well as communicate and cooperate with one another for 
individual, corporate, and national berwfits. 

Hiese growing demands on the work force may explain why the 
knowfedgt and skills of the population, rather than the accomplish- 
ments of the elite, are rej^nsibk? for ea>nomic growth and employ- 
ment. The diligence and skills arhieved by high standards and hard 
wi)ris in language, mathematics, and science in elc eniary and second- 
ary schools, combined with continuing on-ihejt* training, may foster 
ecomimic prcxluaivity more than higher education and .scientific dis- 
cover>'. 

Trained Inc^Mcity? 

In addition, higher education may even be counteqirtxluctive for 
economic growth, as suggested by current data in Figures H I ami 8.2 
and several causal possibilittos. In his 7*et)r>' cftfoe Leisure Class, for 
example, University of Chic^ economist Thorstein Veblen (1899) 
wrote of the "trained incapacity" of the highly educated and the elites 
unprcxiuaive and "conspicuous consumption" of their time and money 
on the useless and esoteric to set themselves apan from the working 
people. 

The continuing distinction of (Abridge and Oxford in the basic 
sciences and humanittes has done little for Britain's plight: social-class 
tensions, unemployment, and ikfclining commerce and industry Uni- 
versity emphasis on the classics and disdain of prartical matters tradi- 
tionally has set social classes apan and divided collegecducated leatfers 
from blue<ollar workers. 

In England, once the mightiest empire and workshop of the world, 
a curriculum theorist wrote that teachers "are right to resist the view of 
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curriculum as being deicmitmxl by the ncetls trf s<Kict> aiul t^sjxt ially 
the needs itf imlusiry ^' (Lawion 19^, p. 134). Is this pan of the English 
disease" thai other Europeans fear? 

Higher education enrollments are also asscKsaied with a shorter 
work week, high rates of divorce, and television consunij-nion, AltlKmgh 
affluence or nKxiemiiy may cause all these indicators, jx^ssibly tlK lei 
sure of the "new class" or die ivory tower of theory encourages cultural 
permissiveness and decline of the work ethic. 

Schocd Achievement and National Welfiswe 

As suggested by mvent lEA and other studies (Walberg I9K3, 
McKnjght 1987, and lEA 1988), the short sihix)! v^ear, the lat:k of cui^icu- 
lum rigor, skm^ repetitive leaching, and little ht>mewx)rf; ard parental 
involvement, seem the major causes of ^hk^t IIS. achievement. Even if 
this were certain, however, reasonaWe pec^plc may disagree on tlu! 
value implicationa Some may see in these trends the continuing ecx)- 
nouiic and cultural dcvline of ilie Wc^t. Others may pertx-ive a reorien- 
iatk>n of stKiety and educatuMi for a new era of libei^ition, leisure, and 
comumption in contrast to the tfeferral of gratifiration to attain k>ng 
term ends (Freud 1920) and prc:>mitience cf the work ethic that has cliap 
acterized the capitalistic Western etionomies since the Protestant Refor- 
mation (Weber 1958). 

Howei^er, high school students a|^parently are unaw^are that il>e lei- 
sured, permissive millennium has arrived. Vtlien asked to name the 
most important American problem in a recent survey for the Natk)nal 
AsscKiation of ikxondary Schcx)! Principals (Heard 1984 ), more than 
half mentioned unempk>yment, and a fifth said inflation, Tliese ranked 
first and second in frcx}uenc>^ among 14 problems menik)ned. When 
asked what il^y mtxst want out of life, more than a quarter answered 
career success, a quaner said happiness, and 21 peaxTit mentioned 
marriage and family — to name the top three, 

Japan's indext^ — nearly three times ilw gixiwih, tmefounh the 
unempbymcnt, and one^third the divortx* rate of the IIS.— are more 
consistent with the expressc*d values of the American youth than are tte 
U.S. indexes. Japan provides an example of the pH)ssible causal link of 
school achievement and economic vitality With the higl>est test scores, 
nearly the highest growth, and nearly the lowest utK»mployment (rf tin? 
nations with complete informatton Japan mav have set the educational 
and economic standards for the rest of the world. 
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Unlike Europe. Japan n: w grjduaies a ga-aicr fraawn of its popula- 
tion from high school than does the U.S. (96 vs, 76 percent, U.S Depm- 
ment of Education 1987, p 79). NtHwiihstanding these often-expressed 
opinion.s, little evitk*mx' shows that Japanese students art? uncreative, 
overstressed, and suicidal because of high standards and homework 
CWalbetg 1983, U.S. IXpanmeni of Education 1987). 

Prospects 

The nK)St recent data on science and mathematics achievement 
offer gkx)my puispeos if Adam Smith and others are right about the 
imp»)rtance of national ability for welfare. In The IJndemdnetmg Cur- 
riculum, McKnight and others (1987) a'ported that, considering both 
test scores and enrollment, U.S. math arhievemeni is "very low." Of Jkh 
grade students in 20 tx)untries. tliase in the U.S. rankt>d 12th in algebra. 
16th in geometry, and 18th (atxjve Swaiuland and Nigeria) in measure- 
ment. Of 12th graders in 15 countries, those in the U5. scored 14th in 
advanced algebra and 12th in elementary funaitias and calculus. 

Nor can we take amfon in our brightest students. Among the top 1 
pcTcent of 12th graders in the 15 countries. U S. students scored worst 
in algebra and exceeded in funaions and calculus only British Colum- 
bia, which dtiesn't offer these subjects in the high school curriculum. 
These findings are consistent with the Natiional Assessment erf Educa- 
tional Ptt)gress ( 1 988) conclusion that 1 .5 million American 17-year-olds 
reach the end of high school each yt?ar unable to reason mathematically 
(p. 17) 

The U.S. dtK-s little better in science (lEA 1988) Although near 
average at gracfcs 4 and 5, U.S. students exceed only those in the Philip- 
pines and Singapore among 17 countries at grackrs 8 and 9. Among 
advanced 12th gr^ students in 14 countries, moreover, those in the 
U.S. ranked last in blok)gy, llth in clwmistry, and 9th in physics, 

in txjndusion. Ix^th the importance and the uncertainty d national 
indexes and their causal relationships deserve emphasis. The U.S. has 
l^ed in economic grov/th. whkh was well prediaed by previous sci- 
ence test scores. We lag even further in the most recent comparisons. 
Hie indexes in which we excel — ctillegc enrollments, leisure, televi- 
sion consumption, and divorce— may not be conducive to future pros- 
perity and national welfare. 

It Ls passibkf, though, that during the first 18 years of life, femilies 
and schools can better ft>sier knowledge, diligence, cooperatlveness. 
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and ottier pswiK>k)gical ami scKial traits that promtm* a higlHT quality 
of future life, Ailht^ugh the correlations of inciexes h;u^dly pro\x' this 
interpretation, the>^ are wnsisteni witli it and with what many citizens 
telieve. We may ignore its implications at our children's and our own 
peril. 
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Afterword 
A World of Assessment, 
A Universe of Data 

CHESTER E. FINN, JR 

Americans ha%'e always apprectatcd international inft>mia- 
tion about education. We have at limes been self-abM^rbed, 
.even insular. But we value it now. Indeed, we crave it. Ever>' 
ttae an lEA repxjrt comes out, for example, petiple are riveted by the 
data We now recc^nlze the need for more, bener, and more regular 
lniemaik>nal information than is currently available. 

Cross-national research widens our horizon and can sometimes 
surest nonincremenial means of improvement. The disawery that a 
number of other countries devote more time to schtx)linf? than we do, 
for ejtan^le, has stimulated domestic ifebate about lengthening the 
schod day and the school year. 

Educational performance in t^her countries provides benchmarks 
for us to use as we wish. Japan, for instance, does something many 
Americans had not thought possiWe. The Japanese graduate an extraor- 
dinarily large proponion of their stuucnLs from secondary schcx)! while 
simultaneously producing a very high average level of academic 
^hievement. This finding challenges our conventional wisdom. Con- 
t -v to the assumption of many American educators, equalit>' and excel- 
lence can both be achieved within the same system. Japan illustrates the 
possibility. 

AnotJier instance in which an iniernationaJ study removetl our 
blinckrs is the lEAs Second International Mathematics Study It con- 



Cbcster £ firm, Jr, is Director of the Educafk)nal Excellence Netu Kfrk in 
msbington, D C, and Prc^essor ofBduuttion arid Public Folic)' in the 
Vdnderbilt University €0ce in Washington, D C 



ChekitoE. Finn, Jr. 



firmed that achiev-ement folkws content awen^e, that students tend to 
learn what is taught. The study also revealed something tliat many 
i^nericans had not supposed passible^that student*; can bt* taught 
oomj^ex mathematics at n relatively early age. 

International compar;;tive data frequently indicate areas that need 
our attention. The 1972 lEA lejKling and literarure studies, for instance, 
showed that a major weakness among U.S. secondary stucfcnLs is com- 
prehension. Findings from the latest lEA science study show U.S. stu- 
dents discoun^inf»ly far behind their peers from other lands in physics 
chemisirv, anc ^rioJc^'. 

lntt.rn;^iional comparisons t)f student performance gener^illy stir 
tfcbate in the United States, and they sometimes trigger aaion at cIh? 
state and kxal levels. In the aftermath of the Second Internal it)nal Math- 
ematics Study, school districts in Marylaml formed a consortium to 
examine fte study results and to initiate reforms An organizatit)n in 
Minnesota, the Mathematics Quorum, informed e»:h sdiool distria in 
the state abciut the study's conclaswns. Three other states— Virginia, 
Florida, and hwa — conduaed their own sun-eys of mathematics to 
compare themselves with ether countries. The results helped Floridiaas 
devetop a new geometry curriculum. 

International information has helf^ fuel effons to .strengthen 
education in the MS. and in other democracies as well Progress in 
social welfare, economic gnjwih, and cultural vitality is something that 
concxTtts all trfus, and progress in each of these domains depends in no 
small part on improvements in education. At a time when lechnolfjgy 
has shrunk the globe— when TV, teleplwnes, satellites, facsimiles, and 
jet travel have turned oncxxlistant lands into neighbors— we need, 
more than ever, to learn from each other, to benefit fn)m each other's 
experiments and experiences, triumplis and errors We need to under- 
take collaborative efforts wterever possibfc. 

Wt' also need more data — and more kinds trf it— on education 
around ilie worid. We need, as one exampie, a reliable way to chan edu- 
cations pea'entage of the gross natM)nal pnxluci in each countr>'. Not 
just government expenditures t)n cduc-dtk)n:«// expenditures on etluca- 
tion. Win can't this be done? It is among the most lundanKfntal points di 
reference for comparing education systems and their relative prcxluctiv- 
ity. Another international blind spot is the curriculum. Can w« not do a 
decent comparison of cx;unse a>nteni, sul^ert by subject, across nations? 

St)me of the reasons we lack international data are understandable, 
in their way even legitimate. Countries organize their systems differ- 
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ently and measure things differently Data on schcKil dn^ms, for 
example, are difficuk enouf^ to coltea within the US. Its hard even to 
get tte 50 states to agree on a ddinitk>n and on the eJements thai 
s^KHiId be measured, Intcmatwnal agreenwnt is that much hanfer to 
reach, m it is vital that we woric out a way to do this. Attrition and com- 
plex* rates are among the essential indicators of any education system. 

If some C3f the reasoas we lack international comparisons are rea- 
sonaMe, at least for the time being, others are not. Too many ccHin- 
tries — the U.S. has been vuIfM?raMe to this at times — have been secre- 
tive, uncooperative, or simply lazy We need joint efforts to galvanize 
support for more valid, nx>re detailed, more varied, ; mi more useful 
international information, 

Tlie international agemry best positioned to ^xxnnpli^ this is th^ 
Organization for Economic Coojx^ration and Development (OECD). 
During my time in the Education Depanment, we pressed OECD hard 
to devekp a tetter set of international indicators, and, in tte past 
years, we have seen promising movement in this dire»:tton, 

In the fall 1986, we lx>sted an International Mt*eting on Eduoi 
tlonal Indicators in Washington untfer OECD aaspices Representatives, 
from most erf the member countries atteiided, many indicating an inter- 
est similar to ours in securing timely, useful information about educa- 
tion quality, performance, and results. 

As a result erf this meeting and others that followed under the ^gis 
of OECD's Educatk>n Committee and its Center on Educational Research 
an<^ Innovation (CERI), OECD has initiated a projea to develop educa- 
u il indicators. TY^ first step was to divicte the universe of possible 
indicators into five categories and invite countries to panicipate in the 
development irf those they are mc»t interested in. 

The United States' greats interest is in cognitive achievement; 
therefore, we volunteered to coordinate the networic on a?gnitive out- 
comes. Our National Center for Education Statistics is currently working 
with a dozen countries in the ctesign ctf a set (rf such indicators and the 
ineans cf gathering tl^ requisite infoimation. 

It seems to me that, instt.-Hi ctf each working in spk?ndid isolation, a 
more stable and closer relationsliip benveen lEA and OECD would be 
most bei^ficial. lEA very likely will be the source of much of the cogni- 
tive outcorr^ data the OECD will want to gather, and it certainly would 
be a wa^ of r^ources if OEQ ) were to replicate what lEA is already 
ctoing. Organizationally, OECD reprc*sents goverrMnents, while lEA 
members represent mt^y research organizations, universities, founda- 
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lions, and some governments. OECD Is a fairiy siable enterprise, 
finaiKcd lai^gely through fixed annual payn^ts from member coun- 
fries. lEA is rather more fragile, highly dependent on private phiian- 
ihrq>y and volunteer labor TTiere is, to my mind anyway, m cAvious 
symbiosis cf interests, caf^biliiies, and strengths waiting to be devel- 

Although the single hk^i important yardsiick of the performance 
of any education system Ls sauctent learning, wc also need data, research, 
and indices d many cnher dimensions— features such as curriculum 
content, school organization, attendance, parent invoh^ement, home- 
worii polici«, retention rates, enrollments in ad\'anced courst^ and in 
college, hyw education revenues are used, and performance in higher 
education. We need to gauge the gaps between the intended curricu- 
lum, the implemented currkiilum, and the attained curriculum. And we 
need to look for links between student outcomes and the variaWes that 
parents and educators ran control. Tm thinking here of Hardd Steven- 
sons* finding that American parents tend to attribute their diiid s learn- 
ing to innate ability whereas Asian parents more crften believe it to be 
mainly a result of hard work This, d course, is pan of tte reason Asian 
^udents learn more. 

Studies such as Stcveason's are doubly imfx)rtant because tliey 
indicate not only differences in student achievement, but also faaors 
that help explain the reasoas for iliase differences. Stevensons study 
points to a variable that lies within human a)ntrol attitudes are some- 
thing we can work to change. 

Vd like to see more .studies suc^h as this one and more analy.ses {)f 
data— across iastilutions, regions, nations, and other gec^graphic and 
cultural boundaries. Such analyses can reveal wt^knesses in an educa- 
tion system. They can also highlight features most susceptible to 
improvement, that is, those most likely, if mcxlified, to yield increases in 
student outcomes. 

Of coun^, not ever>thing ^^v need to find out is amenable to mea- 
surement and certainly not to testing. Ps\r}>ometrics cannot capture or 
gauge subtle qualiMes of mind and spirit. But much that vw want to 
know can be quantiiled and measured, and it should Ik?I The whole field 
of testing and measurement is making great strides and will make more. 
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Although wc must not forgei the immeasurahles, we must slight 
what can be measured. 

Today the uwk of the lEA comprises the single most valuable asset 
in the world with resfHXt to the cross-national appraisal and analysis of 
education outcomes. It is far from a perfect organization, but our vow 
must be to strengthen it, ior only w^ith th< kind of information it pro- 
vides will we finally know^ whether our current reform effons are pro- 
duaive and which areas need our attention next. 
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